{ "id": "2005-may-june-charting-test-cheaters", "slug": "2005-may-june-charting-test-cheaters", "url": "https://dwillis.github.io/uplink/article/2005-may-june-charting-test-cheaters.html", "year": 2005, "month": "May-June", "monthNum": 5, "headline": "Charting test cheaters", "kicker": "EDUCATION", "author_name": "Holly Hacker", "author_title": "Reporter", "affiliation": "The Dallas Morning News", "summary": "An investigation into unusually high test scores at Wilmer Elementary in Texas reveals possible cheating by educators. The analysis shows that scores improved significantly compared to previous years, leading to suspicion of test manipulation.", "keywords": [ "test cheaters", "Wilmer Elementary", "Texas" ], "topics": [ "Education", "Investigative Methods", "Data & Databases" ], "technologies": [ "SPSS", "SAS" ], "full_text": "\"We just worked real hard.\"\n\nThat's how the principal of Wilmer Elementary near Dallas explained her school's astonishing performance on the state third-grade reading test last year.\n\nWilmer finished No. 1 in Texas, ahead of more than 3,000 other schools. Amazing.\n\nA little too amazing, in fact. Standardized test scores are a big deal in Texas, as elsewhere. States and the federal government rate schools based on test scores. In some schools, bonuses or promotions hinge on good results. In past years the state had rated Wilmer low-performing or acceptable, nothing higher. So how could scores climb sky-high in a year?\n\nOur efforts to answer that question turned into a statewide analysis of test scores. We found nearly 400 schools with unusual swings from one test to another, with several cases that pointed to cheating - not by students, but by educators.\n\nIn Wilmer Elementary's case, a state investigation confirmed what our reporting found: Teachers didn't work \"real hard.\" They gave students answers to the test.\n\nWilmer Elementary belongs to the Wilmer-Hutchins school district, where finances, leadership and academics have been in chaos for decades. The district made headlines last summer when it had to delay the opening of a high school because the building had roaches, mold, mildew and a leaking roof.\n\nA host of financial and legal troubles followed. The FBI and Texas Rangers (the law enforcers, not the baseball team) got involved. And then came some good news: high test scores brought an \"exemplary\" state rating to one elementary school and \"recognized\" status to the other two, including Wilmer.\n\nMy colleague covering Wilmer, Josh Benton, didn't believe it. He had heard rumors of cheating before in the district, but nothing concrete. Pointing to Wilmer Elementary's reading scores, he wondered how we could put the dramatic rise into statistical context. Just how likely was it?\n\nEnter SPSS, the statistical software program. I had grown to love and respect SPSS (and statistics, for that matter) at my previous newspaper, the St. Louis Post-Dispatch. We used SPSS to analyze school test scores relative to family income, as several other papers have done.\n\nSPSS and similar programs, such as SAS, also have a wonderful ability to draw pictures of data. You can see stuff with charts and graphs that you wouldn't in rows and columns. So to see how Wilmer Elementary's third-grade reading scores compared to the rest of the state, I plotted each school's scores for 2003 versus the scores for 2004.\n\nEach circle represents a school in Texas that tested at least 30 students. (Smaller numbers can skew results.) The scatterplot shows a trend: Schools do about as well in 2004 as they did in 2003. But see the circle hovering high above everyone else? That's Wilmer Elementary. It went from an average score of 2,240 one year to 2,501 the next. No other school had such a jump.\n\nThen we looked at the scores another way: comparing third-grade reading in 2004 to fourth-grade reading in 2004.\n\nAgain, Wilmer stands out at the top, with the stellar third-grade score but just a so-so fourth-grade score.\n\nBut that wasn't the only school that stood out. Check out the school on the far right, with the near-perfect score for fourth grade but a lousy one for third grade. That's Harrell Budd Elementary in Dallas.\n\nWe ended up looking at reading and math tests, which Texas students take in grades three through 11. (Other subjects are tested only in certain grades.) We examined the scores four ways:\n\nFrom one grade and year to the next, such as third-grade reading 2003 vs. fourth-grade reading 2004.\nFrom one grade to the next for the same year, e.g., third-grade reading 2004 vs. fourth-grade reading 2004.\nThe same grade over two years, e.g., third-grade reading 2003 vs. third-grade reading 2004.\nThe same grade over two subjects, e.g., third-grade reading 2004 vs. third-grade math 2004.\n\nThe analysis went beyond scatterplots and used a statistical tool called regression, which looks for the relationship between two variables. Based on that relationship, regression makes educated predictions. So given a school's score last year, regression determines how well it should perform this year.\n\nIn Wilmer's case, it scored 231 points higher than expected in 2004, based on its 2003 score. To put that in context, most schools scored within 37 points of their predicted score.", "provenance": { "extracted_by": "pre-v2", "extracted_at": "", "verified": true }, "similar": [ { "slug": "1995-april-whats-affecting-test-scores", "score": 0.26 }, { "slug": "2000-march-lake-wobegon-effect", "score": 0.258 }, { "slug": "2006-july-august-finding-diamonds-in-the-rough", "score": 0.246 }, { "slug": "1996-september-count-me-in", "score": 0.23 }, { "slug": "1992-april-mo-education-department-releases-test-scores-to-micar", "score": 0.167 } ], "html_url": "https://dwillis.github.io/uplink/article/2005-may-june-charting-test-cheaters.html", "json_url": "https://dwillis.github.io/uplink/article/2005-may-june-charting-test-cheaters.json" }