Creating reams of school statistics is not difficult in itself. But the more challenging task was our mission to organize the reams to be able to help our readers make educated choices about schools. We also wanted to show intangible qualities of schools with interviews from educators, parents, students and others.
So how did we do it?
The Arkansas Democrat-Gazette published its second annual school statistics report on Aug. 23 in a 24-page tabloid ranking of 73 elementary, 15 junior high and 12 high schools in Pulaski County.
Quantifying performance
The rankings used a broad set of categories to quantify school performance in areas identified by educators, parents and federal courts.
With most categories, the newspaper collected raw numbers then calculated rates - mostly per 100 students - to compare fairly schools of any size. The calculation was simple. For example, to calculate the high school dropout rate, we used the formula: (dropouts/enrollment) X 100. We chose SPSS but could have easily used Excel or Visual FoxPro. The categories:
Elementary schools: rates per 100 students for teachers, advanced-degree teachers, parent-group membership, volunteer hours and suspensions; a percentage showing either crowded or unused classroom space, calculated by enrollment/capacity; average SAT scores for the fifth grade; racial disparity in those test scores (an issue a federal court is monitoring); and spending per student.
Junior high schools: all the elementary figures except the test scores and racial disparity were calculated for seventh-graders; and rates per 100 students for dropouts and expulsions.
High schools: the elementary and junior high school categories except the scores and disparity were for 10th-graders; ACT examination averages; percentages of seniors taking the ACT; percentages of previous graduates who were forced to take remedial courses in state colleges and universities; percentages of seniors taking college preparation courses; and the total number of courses available.
The Democrat-Gazette constructed tables with rankings - that is, which school is the "best" given this set of categories, all of which were valued equally. In 1997, we ranked with Excel. This year, we used SPSS, a statistical software package that is much handier for this purpose.
We identified schools that beat the odds and did significantly better in the test scores than predicted. Then we wrote about how those schools are making a difference.
We had SPSS rank each category in reverse order. That, in effect, created point scores for each school. For example, the worst elementary school in the teacher-student ratio category was ranked "1." The best school, then, would have a score of 73, or the total number of elementary schools.
Then we had SPSS create an average score using all of the scoring columns. The top average score became our top-ranked school. We showed the average scores, but instead of using unfamiliar groupings such as quartiles or quintiles, we assigned letter grades for schools. Those letter grades helped readers understand that average scores were quite close for many schools.
But was it fair? Not if that were all we did. Like last year, and even more in-depth this year, the newspaper showed readers how to zero-in on categories and to find schools doing well in them. Graphics portrayed how schools fared in each category. Readers can track categories they deemed most important and judge how particular schools are doing.
Chasing intangibles
We wrote serious stories about some of the higher-ranked schools and showed how they have been successful. Those stories aren't top-heavy with numbers but rather emphasize interviews. We did stories about some schools, though not ranking highest, that ranked well in specific categories. We also looked at schools that fell in the rankings this year to learn why.
And we went even beyond that, especially with the elementary schools. Using fifth grade average SAT scores and the percentage of children taking part in the free and reduced lunch program, we created a regression analysis showing how the two factors are related. We identified schools that beat the odds and did significantly better in the test scores than predicted. Then we wrote about how those schools are making a difference.
For our purposes, we looked at one other scatterplot. Pleasingly, the plot showed that race had no bearing on schools' average scores.
Most newspapers avoid ranking and grading schools. We rank and grade them because we are confident of our statistics and want to help our readers make sense of the data instead of merely presenting a jumble of numbers that lead to no conclusions.
jeff Porter can be reached by e-mail at jporter@ardemgaz.com