{
  "id": "2002-february-doing-cross-tabs-can-create-quick-comparisons",
  "slug": "2002-february-doing-cross-tabs-can-create-quick-comparisons",
  "url": "https://dwillis.github.io/uplink/article/2002-february-doing-cross-tabs-can-create-quick-comparisons.html",
  "year": 2002,
  "month": "February",
  "monthNum": 2,
  "headline": "Doing cross tabs can create quick comparisons",
  "kicker": "Tech tips",
  "author_name": "Jeff Porter",
  "author_title": "Managing Editor",
  "affiliation": "IRE/NICAR",
  "summary": "Cross tabs and pivot tables are powerful data analysis tools that allow journalists to quickly reorganize and compare large datasets by rearranging fields into rows and columns. The article demonstrates how to use SQL queries in Visual FoxPro to create cross-tab comparisons of SBA disaster loan data by state and loan status.",
  "keywords": [
    "spreadsheet",
    "database",
    "comparison"
  ],
  "topics": [
    "Data & Databases",
    "Investigative Methods",
    "Technology"
  ],
  "technologies": [
    "Excel",
    "FoxPro",
    "SQL"
  ],
  "full_text": "Doing cross tabs can create quick comparisons\n\nBy Jeff Porter, IRE/NICAR\n\nCall it an art form - rearranging data in a useful way, comparing fields across columns instead of down long rows. The spreadsheet program Excel calls it a pivot table. Database managers often call it a cross tab. For Visual FoxPro users, it's a fast, quick step to compare hundreds of thousands of records instantly.\n\nTake, for example, Small Business Administration disaster loan data. The loans themselves are the primary form of federal assistance for nonfarm, private-sector disaster losses. The loans help homeowners, renters, businesses of all sizes and nonprofit organizations finance their rebuilding. The loans, which are repaid to the Treasury, are provided at low-interest rates and have tailored repayment schedules befitting each borrower's financial capability.\n\nTo create a cross-tab comparison, the rules are simple.\n\nYou want three fields. Ideally, the first has the most potential variables. Using the SBA disaster loan data, we'll compare states and a few other geographic areas, such as territories.\n\nIn the second field, you want to limit the number of variables, technically, to 255. In practical terms, you want just a few variables. In the SBA disaster loan database, there's a field called CHGOFF, noting whether a business disaster loan is deemed uncollectible. It's either a YES or NO - just two possible variables and perfect for a cross-tab.\n\nThe third field is the data field, a total count or sum, for example, categorized by the two other fields. For this example, the count will be categorized as the status of each loan in each state.\n\nFor Visual FoxPro 5.0, here's the SQL to first list the borrowers' states, whether the loan was charged off, and the total number for each category by state. The query limits the type of loan to just businesses - coded as a capital B in the data not individuals, because the SBA won't tell the status of an individual's loan. In this example, the number of loans is limited to those since the beginning of 1990 to businesses: select borstate, chgoff, count (*) from disaster; where apprdate > {12/31/89) and type = 'B', group by 1, 2 The results would look like this:\n\nTHE Query\n\nBorstate Chgoff Ltd\n\nNO 1\nAE NO 1\nAK NO 68\nAL NO 97\nAL YES 2\nAR NO 216\nAZ NO 10\nCA NO 150\nCO NO 25\nCT NO 20\nDC NO 37\nDE NO 4\nFL NO 829\nFL YES 4\nGA NO 161\n\nHelpful, but only slightly. Now, one could export results to Excel and create a pivot table to show the CHGOFF variables across the top of the spreadsheet and the states down the side, but Visual FoxPro can do it quicker. Just use one line of SQL in the software's command window: do _genxtab)\n\nWith this short command, you've created a new table called Xtab, created in the software's default directory. It should, though, be already open, so simply type\n\nbrowse\n\nAnd you should see this:\n\nHE Xtab\nBorstate No Yes\n 28 4\nAA 2\nAE 8\nAK 695 29\nAL 1416 117\nAP 4\nAR 642 25\nAS 179 41\nAZ 340 38\nCA 36787 4752\n\nBetter - at least one can read across and see in a glance how a particular state is doing. Note here, for example, that in California, 36,787 loans aren't in trouble, while 4,752 are categorized as uncollectible.\n\nWhat's that as percentage? Again, one could export over to Excel; instead, though, one could run one last, quick query to show the result. For this example, the number of total loans is limited to 101 or more by state or territory, to leave out areas with a tiny number of loans that might skew a comparison: select borstate, no, yes, yes/(no+yes) pct from xtab; where no+yes > 100; order by 4 desc\n\nThe query creates a new field - the number of loans charged off, divided by the total number - and names it PCT, short for percentage. Also added: The last line tells FoxPro to sort the records by highest percentage to lowest, finding the states with the highest percentage of loan default. The answer:\n\nQuery\nBorstate No Yes Pct\nAS 179 41 0.1864\nRI 881 18 0.1538\nSC 1770 303 0.1462\nGA 2007 289 0.1259\nPR 5349 732 0.1204\nFL 10345 1394 0.1187\nCA 36787 4752 0.1144\nHI 1666 207 0.1105\nMO 1922 221 0.1031\nIL 2074 237 0.1026\nNH 132 15 0.1020\nAZ 340 38 0.1005\nLA 2367 251 0.0959\nKS 642 61 0.0868\nNM 139 13 0.0855\nSD 718 67 0.0854\nIN 410 38 0.0848\n\nIn the first record, AS stands for American Samoa, a U.S. territory with 19 percent of the loans defaulting during the time period. Those California numbers show up at No. 7 in this calculation, with 11 percent.\n\nStates, of course, make up a simple example. Reporters can filter and cross-tab by city, zip code, even type of business to compare and calculate, filtering before or after running the cross-tab command. The SBA disaster loan data table contains 731,352 records, far beyond Excel's pivot table capability.\n\nOf course, even a well executed cross-tab doesn't relieve the reporter of reading the fine print about the data. The table lists the state of the borrower, not necessarily the location of the damaged property. The majority of the most recent loans, including those in the days after the Sept. 11 attacks, haven't had time to run into serious trouble.\n\nJeff Porter can be reached by e-mail jeff@nicar.org",
  "provenance": {
    "extracted_by": "anthropic/claude-sonnet-4-6",
    "extracted_at": "2026-07-05",
    "verified": true
  },
  "similar": [
    {
      "slug": "2005-january-february-disaster-aid",
      "score": 0.174
    },
    {
      "slug": "2006-january-february-disaster-loans",
      "score": 0.164
    },
    {
      "slug": "1996-august-the-rich-get-richer",
      "score": 0.141
    },
    {
      "slug": "1997-may-cheating-at-foxpro",
      "score": 0.134
    },
    {
      "slug": "2002-february-flipping-a-table",
      "score": 0.131
    }
  ],
  "html_url": "https://dwillis.github.io/uplink/article/2002-february-doing-cross-tabs-can-create-quick-comparisons.html",
  "json_url": "https://dwillis.github.io/uplink/article/2002-february-doing-cross-tabs-can-create-quick-comparisons.json"
}