{ "id": "1997-april-getting-back-to-basics", "slug": "1997-april-getting-back-to-basics", "url": "https://dwillis.github.io/uplink/article/1997-april-getting-back-to-basics.html", "year": 1997, "month": "April", "monthNum": 4, "headline": "Getting Back to Basics", "kicker": "For the Novice", "author_name": "Neill A. Borowski", "author_title": "", "affiliation": "The Philadelphia Inquirer", "summary": "This guide explains how to properly analyze population data in computer-assisted reporting, emphasizing the importance of visualizing data through plots and calculating basic descriptive statistics like mean, median, mode, and range rather than over-applying complex statistical tests.", "keywords": [ "statistics", "analysis", "data" ], "topics": [ "Data & Databases", "Investigative Methods" ], "technologies": [ "Excel" ], "full_text": "At the outset, we have to agree on something: Much of the time we really aren't using \"statistics.\" We're mostly in the business of measuring populations. This distinction is important. Many statistical tools are designed to help the researcher use samples to make inferences about the population. In computer-assisted reporting and analysis, we usually have the population in front of us. There's no guessing what the full universe of data looks like. We have it all: Every teacher's salary in our state, every fatal traffic accident in the nation, or every sentence imposed by a judge. By over-testing our data, we may be complicating and confusing our analysis. Avoid using statistical tools \"because they're there.\" Always keep it simple. With that said, here are some fundamental steps to measure your data. Let's say our database is all of the teacher salaries in a state:\n\nAlways, always plot your data\n\nThis probably should be your first step after making sure the database is clean. Plot a histogram (for a description of a histogram, see the page margin to the right). It will show how your data are distributed. A normal distribution will be mound-shaped. Look for oddities in your data. Are there a large number of salaries far off to the right of your histogram? This could be a highly paid district.\n\nCalculate the 'basics'\n\nThese are the range, arithmetic mean, the median and the mode (remember \"ramm\"). Lets say we're looking at salaries. The mean — often called the average — is the sum of the salaries divided by their count. The median is the salary in the middle after the salaries are ranked (when there is an even count of salaries, there will be two salaries in the middle; the mean of these two salaries is the median). The mode is the salary that appears most often. The range is the highest salary minus the lowest salary.\n\nRank, rank and compare\n\nWho gets the most? Group the districts and calculate their averages and medians. Which districts pay the most? How do these compare with the state average?\n\nUse the total experience (in years) of each teacher to calculate the salary per experience year. Rank these. Do you have the salary from five years ago? Calculate the percentage change and rank the change. What district salaries changed the most? The least? For me, one of the handiest tools in software is the =rank function in MS Excel. It will tell you where things stand in a flash.\n\nLook at the spread\n\nTwo sets of data could have the same means, but would look far different when they are plotted. The average may be $45,000 for two sets of salaries. However, one may have a low salary of $30,000 and a high salary of $65,000. The other data set may have a low of $8,000 and a high of $90,000.\n\nThe range will help. But the standard deviation is a better tool to help with this. Keep in mind that there are two standard deviations — the sample and the population standard deviations. (In Excel, the sample standard deviation function is =stdev and the population standard deviation function is =stdevp).\n\nThe standard deviation offers a quick view of how the data are distributed around the mean. Rule of thumb: The mean plus and minus one standard deviation includes about 68% of the data and the mean plus and minus two standard deviation includes about 95% of the data. What are the extreme values or outliers? Check out how many salaries are beyond three standard deviations. How many go beyond two standard deviation? These may be newsworthy.\n\nStudy the scatterplot\n\nWe're not explaining regression analysis in this article (regression looks at the impact one variable, such as experience level, has on another variable, such as salaries). However, you can quickly look at the \"big picture\" through a scatterplot.\n\nThe \"x\" (horizontal) axis is the independent variable — in this case, total years of experience for each teacher. The \"y\" (vertical) axis is the dependent variable — the salary of each teacher. A scatterplot will show you the regression line and the data points scattered about the line. The points farthest from the line are outliers.\n\nA regression line in a scatterplot of teacher salaries would be upward sloping — it would go from the lower left (low experience/low salary) to the upper right (high experience/high salary). If there are points in the upper-left quadrant of your plot far from the line, it might indicate teachers who are paid a lot with very little experience. And if there are points in the lower-right quadrant, it might indicate teachers with a lot of experience but at low pay.\n\nNeill Borowski can be reached at (609) 779-3884, or send email to neill.borowski@phillynews.com", "provenance": { "extracted_by": "anthropic/claude-sonnet-4-6", "extracted_at": "2026-07-05", "verified": true }, "similar": [ { "slug": "1996-april-statistics-resources-online", "score": 0.179 }, { "slug": "1997-november-playing-with-statistics", "score": 0.177 }, { "slug": "1998-december-on-the-internet", "score": 0.154 }, { "slug": "1999-september-returning-to-regress", "score": 0.154 }, { "slug": "1994-october-calculating-medians-in-paradox", "score": 0.153 } ], "html_url": "https://dwillis.github.io/uplink/article/1997-april-getting-back-to-basics.html", "json_url": "https://dwillis.github.io/uplink/article/1997-april-getting-back-to-basics.json" }