{ "id": "1997-june-head-of-the-class", "slug": "1997-june-head-of-the-class", "url": "https://dwillis.github.io/uplink/article/1997-june-head-of-the-class.html", "year": 1997, "month": "June", "monthNum": 6, "headline": "Head of the Class", "kicker": "Stats from the road", "author_name": "Sarah Cohen", "author_title": "On The Road trainer", "affiliation": "Arkansas Democrat-Gazette", "summary": "The Arkansas Democrat-Gazette developed a straightforward methodology for ranking schools by combining multiple quality indicators like test scores, teacher training, and diversity without complex statistical techniques, finding that top-performing schools weren't necessarily in wealthy areas.", "keywords": [ "schools", "ranking", "methodology" ], "topics": [ "Education", "Data & Databases", "Investigative Methods" ], "technologies": [ "Excel" ], "full_text": "When the Arkansas Democrat-Gazette set out to measure the quality of its schools, it didn't look to fancy statistical techniques to do its work.\n\nInstead, reporters and editors thought through the project, read about school reports, evaluated the information they could get and culled the reporting of three experienced education reporters.\n\nThen they came up with a simple set of criteria that they would use to define the word \"best\" for their readers.\n\nYes, test scores are good measures of a school's success.\n\nBut when parents look at schools, they often look also at the environment, diversity and other measures of quality.\n\nDemocrat-Gazette reporters took that into account when devising their index of excellence in schools.\n\nThey included measures of teacher training and coverage, racial disparity, performance, parent activity, and motivation. And they combined them without the regression techniques that have been used elsewhere to find over- and under-performing schools based on poverty.\n\n\"We may do that in the future,\" said Jeff Porter, the computer-assisted editor who crunched the numbers for the series. \"But what was interesting was that the elementary and junior high schools that were at the head of the rankings weren't in the wealthy parts of town. The elementary school at the top is almost a rural school, on some road that kind of meanders around west of town.\"\n\nBeginning at the beginning\n\nThe effort shows why thinking through a project as perilous as the annual school reports is just as important as learning advanced statistical techniques.\n\nEducators tell us there's no way we'll compare schools fairly. But if our job is to provide our readers and viewers with information, not hide it, then our other job is to do it as responsibly as possible.\n\nThat's where Democrat-Gazette's indexing technique comes in. In Excel, Porter essentially ranked each indicator, then added up the ranks. The school with the biggest total came out on top.\n\nThe only numbers trick he played was to turn the rankings upside down: The school with the \"best\" performance got the lowest rank, giving it the most points toward the total score.\n\nThe school with the \"worst\" performance got the highest rank - the No. 1 we always think of in rankings - contributing the least to the total score.\n\nCombining upside-down rankings of many indicators is one way reporters can cut the school reports down to size.\n\nThe approach seems embarrassingly simple for those who have struggled with the numbers morass of school reports.\n\nLots of indexes are done this way.\n\nMost business \"Top 100\" lists are driven by these kinds of rankings. So are the \"best places to live\" indexes.\n\nEven U.S. News & World Report, which ranks institutions from hospitals to colleges, uses a system something like this.\n\nIt might be more sophisticated, but the basic idea is the same - figure out what is important, assign a score for each element, and come up with a total that puts everyone on the same footing.\n\nIs an index made up of rankings scientific?\n\nNot entirely. Does it have drawbacks? Certainly.\n\nYou've lost a lot of information that you'd retain if you used z-scores instead.\n\nAll of these indexes, though, use some combination of judgment, reporting and sometimes even polling involved in deciding what's important.\n\nSome words of caution:\n\nFigure out your indicators carefully.\n\nIf you decide that racial disparity is a bad thing for schools, then you have to find some indicator of this that will treat all schools equally. Democrat-Gazette reporters had criteria laid out by the court system to decide how to measure it.\n\nThe lesson is clear - find some reputable and impartial source for your standards whenever possible.\n\nAdjust most of your numbers for the number of students. This is the \"per capita\" calculation that NICAR classes harp on so often. Arkansas used many measures \"per 100 students\" instead. It's the same thing.\n\nMake adjustments that educators say matter, but only when they do.\n\nMany school report stories use some kind of adjustment for poverty, often using the percent of the students who qualify for free or reduced-price school lunches.\n\nYou might want to give schools with high poverty rates (or other difficulties like English language skills or high migration rates) a leg up in your ratings.\n\nBy focusing on more than just test scores as a measure of quality, Democrat-Gazette reporters and editors didn't need to make these kinds of statistical adjustments.\n\nThrow out redundant indicators or others that don't tell you much.\n\nSchool systems often throw out reams of information, much of which is useless in judging the quality of schools.\n\nFor instance, New York's Web site scores of indicators, some of which are nothing more than the percentage of kids who passed competency tests in each subject for each grade. This doesn't help when they're always 99 or 100 percent.\n\nDecide what you'll do if there's a lot of missing data.\n\nIn Arkansas, like many other places, reporters simply couldn't get reliable information for some grades.\n\nYou may decide to focus only on the very early grades, then junior high schools.\n\nYou may have to skip a story on middle schools if there aren't enough to make a fair ranking.\n\nOr you may decide to include an average of the schools' other indicators as an estimate. Just don't let the missing values sit there, though. It will pull down the scores of otherwise fine schools.\n\nCollect raw numbers, not rates. Democrat-Gazette reporters wanted to make sure each indicator was calculated the same way.\n\nSo instead of collecting the usual drop-out rates, reporters had to ask school officials for the raw number of dropouts.\n\nThis is different, and raises the ire of some school officials, Porter said. But it's sometimes the only way to make sure your numbers are all calculated the same way.\n\nOnce you've gotten your indicators, though, making an index of rankings is straightforward. Get your final indicators into Excel, in any order.\n\nNow use the RANK() function to assign the scores for each indicator. Then add up all the scores. You have to figure out which direction is \"good\" and which direction is \"bad\" for each indicator.\n\nFor instance, a lower teacher-student ratio is usually considered better than a higher one. So a low number is \"good.\"\n\nOr a higher test score is better than a lower one, so a high number is \"good.\"\n\nHere's how a formula might look for a \"high is good\" indicator.\n\nThe RANK() function requires three pieces of information: the number you want ranked, or this test score; the group you want it ranked within, or the entire range of test scores; and the direction you want it ranked.\n\nA \"1\" means that a higher number will end up higher.\n\nA \"0\" means that it will rank normally, with a lower number assigned to a high score.\n\n=RANK(B3,B$3:B$77,1)\n\nNote the dollar signs.\n\nThey mean that, as you copy down, the formula will remain correct. But they also mean you can copy it across, only changing the \"1\" at the end when you come to an indicator in which \"low is good.\"\n\nTo see the Arkansas Democrat-Gazette's full story about the quality of Arkansas schools, go to www.ardemgaz.com\n\nOf course, using more sophisticated techniques can help you find the stories in your school system.\n\nIn the July issue of Uplink, we'll use the statistics column to profile a set of stories that used statistical techniques to find schools that overcome difficulties well, or sit on their advantages.\n\nSarah Cohen can be reached at (301) 942-2199 or e-mail her at sarah@nicar.org", "provenance": { "extracted_by": "anthropic/claude-sonnet-4-6", "extracted_at": "2026-07-05", "verified": true }, "similar": [ { "slug": "1998-august-report-cards-revisal", "score": 0.329 }, { "slug": "1998-august-car-cooking-raw-numbers", "score": 0.232 }, { "slug": "1997-august-inexperience-reigns", "score": 0.203 }, { "slug": "2000-september-school-test-scores", "score": 0.195 }, { "slug": "2006-july-august-finding-diamonds-in-the-rough", "score": 0.195 } ], "html_url": "https://dwillis.github.io/uplink/article/1997-june-head-of-the-class.html", "json_url": "https://dwillis.github.io/uplink/article/1997-june-head-of-the-class.json" }