{
  "id": "1997-decemberjanuary-error-free-goof-proofing-car",
  "slug": "1997-decemberjanuary-error-free-goof-proofing-car",
  "url": "https://dwillis.github.io/uplink/article/1997-decemberjanuary-error-free-goof-proofing-car.html",
  "year": 1997,
  "month": "December/January",
  "monthNum": 12,
  "headline": "Error Free Goof-proofing CAR",
  "kicker": "",
  "author_name": "Ronald Campbell",
  "author_title": "reporter",
  "affiliation": "The Orange County Register",
  "summary": "A journalist recounts nearly making a critical mathematical error in analyzing mortgage data, revealing how computer tools can mask journalists' fundamental weakness with numbers and statistical analysis.",
  "keywords": [
    "mathematics",
    "journalism",
    "error"
  ],
  "topics": [
    "Investigative Methods",
    "Data & Databases",
    "Journalism & Newsroom"
  ],
  "technologies": [
    "FoxPro"
  ],
  "full_text": "Confession is good for the soul. Last May, I very nearly made the most embarrassing error of my 21-year career. The only thing that saved me from printing the error was a skeptical remark from one expert. The only reason I made the mistake in the first place was because I got careless using a computer. Computers have done wonderful things for journalism. At the same time, however, computers have exposed a basic flaw in journalists: We are, most of us, terrible at math. We became reporters because we couldn't cope with chemistry or economics or, God save us, engineering. On our worst days we struggle to remember how to calculate a percentage or a percentage change; some of us don't know the difference.\n\nYet computers invest us with the illusion that we have mastered numbers. Senator Sleeze collected $5,000,000 from 20 tobacco-industry groups; the computer says so. The crime rate is higher in North Palooka than in Detroit; the computer says so. The city's five top banks made no loans in the inner city; the computer says so.\n\nIn fact, the computer has said none of those things. We have said those things, and any of a number of errors could have led us astray.\n\nError #1:\n\nNominating the wrong denominator. Every percentage implicitly is a comparison of one value with another. Pick the wrong denominator (the number by which the numerator is divided), and chaos results.\n\nPerhaps the most common example of this error in CAR comes each year with the release of FBI crime statistics. Crime rates are calculated by dividing the number of crimes in a city, county or state by the number of people living there; this number is then recast as a rate per 10,000 or 100,000 to allow easy comparison among cities or counties. The problem comes when you've got, say, 1990 population estimates for one city and 1995 population estimates for another.\n\nI stumbled into a subtle version of the same mistake. While analyzing Home Mortgage Disclosure Act data for California last spring, I decided to calculate each major lender's market share of the loans originated for whites, Hispanics, Asians and blacks. After a few hours of coaxing these numbers out of FoxPro, I reached a stunning conclusion: While the top 10 lenders originated 30-40 percent of all mortgages to whites, Hispanics and Asians, they originated just 13 percent of all loans to blacks.\n\nAs a result, my analysis suggested, blacks were unusually dependent on expensive \"sub-prime\" lenders.\n\nLet me pause for a moment and admit that after four or five years of HMDA diving, I had grown quite bored with my annual mortgage story. Here, then, was a discovery worth savoring: a national story buried in a familiar piece of data. Every expert with whom I broached the subject — academics, analysts, even bankers — expressed surprise and enormous interest.\n\nAnd then I ran across an expert at the Mortgage Bankers Association, an economist who spent even more time obsessing on HMDA data than I did. He didn't believe my results. With a certain amount of self-assurance, I offered to redo the analysis while he waited. Within a minute or so I had my numbers: My original analysis was flat-out wrong; the top 10 lenders had the same share of black mortgages as they did of whites, Hispanics and Asians.\n\nLater, I redid my calculations; as near as I could tell, my original mistake occurred because I had divided the top 10 lenders' total volume to black borrowers by the overall loan volume to Asians - the group I had analyzed just before doing the analysis of lending to blacks.\n\nError #2:\n\nWhat's less than nothing? The answer, of course, is nothing. But with just a bit of work you can calculate a percentage change showing that a quantity has dropped 150 percent - a remarkable feat, but one that routinely makes it into the newspaper. The easiest way to do this is to compare a negative and positive number, as when a company reports a profit after a string of losses.\n\nWhile preparing this article, I posted a message on the NICAR list, inviting anonymous confessions. Here's one:\n\n\"Early in my use of spreadsheets, I did a percentage change formula backwards for a 50-state list. The scary part: It sailed right past two copy editors, even though it said some states' pollution had dropped by more than 100 percent. Now I hand-check formulas on a calculator, and people in our group double-check each other before we release calculations.\"\n\nError #3:\n\nComparing the incomparable. A murder is not the same as a car theft or a robbery. So why does anybody ever compare cities' crime rates without at least breaking down the rates between violent and non-violent crime? And why does any editor sit still for such journalistic malpractice?\n\nError #4:\n\nMissing the obvious. I hadn't intended to write a screed about the FBI Uniform Crime Report, but my data is leading in that direction. Another anonymous confession:\n\n\"I was doing the FBI Uniform Crime Report thing, ranking California cities by crime rate. When I was all done, I started calling all the usual suspects for reaction. One cop-management guy said, 'hmm,' or something, then 'What's the top 10?' I read them. He said, 'hmm,' again, then 'Where's Oakland?'\n\n\"Well, the long and short of it is that, as you probably can guess, Oakland didn't get its data in on time and wasn't on the list, and of course the FBI doesn't bother to point this out.\"\n\nSo what is a conscientious reporter to do? It seems to me there are a few good answers:\n\nFirst, if any finding takes your breath away, then breathe in again, hold your breath for a good 30 seconds, let it out - and recalculate your finding from scratch. Do it slowly. Look at your own numbers the way you look at a particularly squirrelly pol's conflict of interest form. Second, widely share your findings before publication. Third, don't use statistical methods that you can't explain while nursing a hangover. I am very comfortable with percentages, percentage changes, medians, means, modes and margins of error. But I seriously doubt that I could explain a chi-square or another sophisticated measure of statistical significance under the best of circumstances. So I avoid using them in my research. I might use them someday - after I've finally finished reading those two big statistics books I've got in my \"to read\" pile at home.\n\nRonald Campbell can be reached at (714) 664-5030 or by e-mail at Ron_Campbell@link.freedom.com",
  "provenance": {
    "extracted_by": "anthropic/claude-sonnet-4-6",
    "extracted_at": "2026-07-05",
    "verified": true
  },
  "similar": [
    {
      "slug": "2002-mayjune-readmetxt-2",
      "score": 0.17
    },
    {
      "slug": "2002-mayjune-readmetxt",
      "score": 0.14
    },
    {
      "slug": "2002-mayjune-readmetxt-3",
      "score": 0.13
    },
    {
      "slug": "1992-october-more-errors-surface-in-hmda-data",
      "score": 0.126
    },
    {
      "slug": "2001-julyaugust-teaching-car",
      "score": 0.122
    }
  ],
  "html_url": "https://dwillis.github.io/uplink/article/1997-decemberjanuary-error-free-goof-proofing-car.html",
  "json_url": "https://dwillis.github.io/uplink/article/1997-decemberjanuary-error-free-goof-proofing-car.json"
}