{
  "id": "1998-july-stats-from-the-road-sampling-not-a-dirty-word",
  "slug": "1998-july-stats-from-the-road-sampling-not-a-dirty-word",
  "url": "https://dwillis.github.io/uplink/article/1998-july-stats-from-the-road-sampling-not-a-dirty-word.html",
  "year": 1998,
  "month": "July",
  "monthNum": 7,
  "headline": "STATS FROM THE ROAD Sampling: Not a dirty word",
  "kicker": "",
  "author_name": "Sarah Cohen",
  "author_title": "",
  "affiliation": "NICAR",
  "summary": "Journalists investigating health care quality can use sampling techniques to efficiently explore large databases and paper records before committing to full data collection, testing whether records contain usable information and identifying what details are worth pursuing.",
  "keywords": [
    "sampling",
    "database",
    "records"
  ],
  "topics": [
    "Investigative Methods",
    "Data & Databases"
  ],
  "technologies": [],
  "full_text": "In a project on the quality of health care, reporters were interested in the details of malpractice claims. There were something like 50,000 payments to victims of doctors, hospitals and other health care providers in a state database. However, the computer records included only summary information. Paper records were the source for more details, such as the nature of the injury and how it happened.\n\nThis problem often arises when working with documents, especially court records. You can find docket information on bulletin boards or other electronic means. But details ranging from previous criminal records to prosecutor recommendations simply don't exist electronically.\n\nThese are perfect candidates for sampling in the early stages of a story.\n\nSampling situations\n\nMost editors and reporters would insist that when the story runs it ought to include all instances. They argue it simply worth the effort to gain more credibility with readers and viewers. But you can sample early on to find out how many and what kinds of records will - or more importantly won't your story.\n\nYou may even find early in a story's planning that the effort probably won't result in a credible The records are too muddled to be of much help.\n\nLarge databases are another candidate for sampling in the newsroom. I think most computer-assisted reporting experts would acknowledge that they've at least once waited hours for a query to run - sometimes overnight - only to find that they left out a field or forgot to include an important case.\n\nHere are some considerations to think about when you begin a project and want to consider sampling as a way to make your work more efficient. Some are geared at figuring out what you need to know before you move further into the deep, dark hole of a long-term story. Others help you figure out whether you need scientific sampling techniques.\n\nCollection difficulty\n\nSometimes, sampling has little place simply because it doesn't save you much. Collecting all of the records on OSHA violations or fatal accidents, for instance, is probably easier than sampling. There's no reason to revert to a small set of records when you can just as easily test all of them. But if you simply can't get all of the records, such as the details of thousands of court cases, then you'll need scientific sampling methods before you can say anything about what's in them as a group.\n\nPassing acquaintance\n\nWhen you're dealing with documents you've never worked with before, you'll probably want to start out with a small group of them to test your database. I usually don't recommend bothering with scientific sampling techniques here. Instead, consider typing in the 50 most recent and 50 from the earliest period you're considering. This will give you three legs up on the project.\n\nFirst, you'll see whether you have the skill (or whether it's even possible) to create queries or other analyses that will address your questions from the database the way you've designed it. Too often, reporters call after spending months typing in records without making this simple test. It makes their work much harder.\n\nAnother advantage is that you'll see how the paper records have changed over time. If you're compiling records from many local offices, though, you'll need to take another step: gathering 10 or so from a few different far-flung districts. This will show you how they vary in quality and in content from place to place, as well as over time.\n\nFinally, you'll know what difficulties you will find. In the malpractice example, for instance, reporters who had stopped early in the project and looked at the most recent records would have noticed the fatal flaw. Malpractice lawsuits take years to settle. Even the most recent records were too old to be considered \"news.\" They could have stopped before they even started.\n\nGetting comfortable\n\nHow comfortable are you with the analytic techniques you plan to use? We've all had the experience of exploring a new data set, only to wait minutes, hours or even overnight for queries that didn't show us anything of value. Consider testing your queries and analysis on a small set of records.\n\nIn this case, you'll usually want something at least moderately scientific. The reason is that you want some hint of what you'll find and, simultaneously, a list of possible problems to check. Do different data entry clerks (or inspectors) enter the data differently? Are fields you care about usually filled out? And what is the scale of the numbers you're expecting to find: one out of a million, or one out of a hundred?\n\nWhat could compare?\n\nDo you have any basis for comparison? Make sure, early on, that you've thought through a few possible leads for your story. It doesn't matter what your suspicions are, just how you'd like to word them if they prove true. Look for words like \"more than\" or \"compared with\" in that short statement. For example, imagine you're examining trends in people who return to prison over and over again. You want to know if they're more likely to test positive for drugs, be in a gang, or be of a certain ethnic group than other felons.\n\nThink carefully about the words \"more likely\" in that sentence. The biggest mistake in building databases - whether they're sampled or not - is ignoring this crucial issue for the paper. It implies that you need to test not only the group you care about but everyone else as well. Make sure, early on, that you could write this sentence based on the records you plan to acquire or the slice of a database you plan to focus on. Sampling will help when you know you can't write the sentence based on your test.\n\nOnce you've decided to sample, the next step involves figuring out how to sample a set of data. We'll tackle that topic next month.",
  "provenance": {
    "extracted_by": "anthropic/claude-sonnet-4-6",
    "extracted_at": "2026-07-05",
    "verified": true
  },
  "similar": [
    {
      "slug": "1998-august-stats-from-the-road",
      "score": 0.295
    },
    {
      "slug": "1993-march-adventures-in-sampling-using-brute-force-to-compensate-for-your-ignorance-about-statistics",
      "score": 0.281
    },
    {
      "slug": "1998-november-census-preparation",
      "score": 0.188
    },
    {
      "slug": "1997-march-growing-collection-of-federal-databases",
      "score": 0.154
    },
    {
      "slug": "1998-june-paper-riches",
      "score": 0.149
    }
  ],
  "html_url": "https://dwillis.github.io/uplink/article/1998-july-stats-from-the-road-sampling-not-a-dirty-word.html",
  "json_url": "https://dwillis.github.io/uplink/article/1998-july-stats-from-the-road-sampling-not-a-dirty-word.json"
}