Posted Monday, April 4th, 2011 by Danny Rosenthal
Don’t Make Education Policy Based on Cheaters
Teachers at a school in D.C. probably cheated on standardized tests. Does that mean that then-Chancellor Michelle Rhee’s* education program is fatally flawed and that Rhee has lost her credibility as an advocate for reform? Of course not. The exposé published by USA Today confirms the need for strong measures to catch cheating teachers. But we shouldn’t abandon efforts to measure student progress or make education decisions based on data.
But first things first. The heated debate over this issue has obscured the underlying facts. Just as Rhee’s supporters shouldn’t reflexively dismiss the story, critics shouldn’t assume that the article provides support for their view that Rhee is misguided, negligent or dishonest.
The documents relied on by USA Today are helpfully collected here, and they are worth a look. They indicate that cheating may have occurred in D.C. but that the school system took allegations seriously. Many have seized on USA Today’s statement that Rhee “balked” at a request from the DC State Superintendent to investigate cheating. A different story emerges from the actual response of the district’s data chief, available on pages 17 to 19, and subsequent reports by the testing company (pages 20 to 22) and an independent investigator hired by the district (pages 106 to 108).
In retrospect, the district probably should have investigated cheating more vigorously. And Rhee’s brusque response to the newspaper last week was also a mistake, which she corrected in a later statement. But there’s been no indication that Rhee or anyone in the district office acted dishonestly. The district leadership concluded that a more extensive investigation based on the available data would have been unjustifiably expensive, intrusive, or unfair to teachers and administrators.
Finally, the evidence for widespread cheating is far less overwhelming than some have suggested. USA Today mentions but does not emphasize that two outside analyses recommended against concluding that cheating had occurred. The article focuses on one school, Noyes Education Campus, and presents two main types of evidence for cheating at that school. First, there were high rates of “Right-to-Wrong Erasures,” in which incorrect answers were changed to correct ones. This evidence is highly suggestive but not conclusive. Second, the article highlights suspicions from a Noyes parent and former teacher — both occurring before Rhee’s tenure.
Outside of Noyes, USA Today’s evidence is much less clear. For example, the article notes that more than half of D.C. schools had an above-average erasure rate at least once in the last three years, a claim that Diane Ravitch repeated in slamming Rhee on The Daily Beast. But think about that for a second: If erasure rate were entirely random, and not a product of cheating, we would expect about half of schools to have an erasure rate above the district average each year. And over three years, we would expect this to happen to almost every school at least once. The article also notes that classrooms in many schools were “flagged” for investigation, but it’s not clear what this means.
With that series of disclaimers out of the way, let’s now suppose that there was cheating at Noyes and maybe even other schools in D.C. What should we make of that?
Cheating at a few schools simply doesn’t demonstrate that testing does not yield useful data or that test results should not be tied to rewards or penalties. Critics have implied that there is no way to administer tests in such a way that cheating will be minimal enough to make results reliable. But that’s a colossal a leap from the evidence in this story. A few cases do not prove a rule. In fact, cheating is already rare and could be reduced even further through better procedures.
Of course, the story also doesn’t prove that testing and accountability is the right approach to reform. We’ll have to save that debate for another day. In the meantime, the story shows that cheating can be a challenge for standardized testing. Indeed, it can be a challenge in almost any situation where incentives are linked to performance (think Barry Bonds). But that doesn’t mean we should abandon the project of measuring progress and acting based on data. It means that we should take every possible step to ensure that our data is fair and accurate.
* The author interned for the D.C. Public Schools while Michelle Rhee was chancellor.
3 Comments
Post a comment
HIgh stakes tests have a history and are continuing to make news. Consider the present scandal in Atlanta.
Education is not different from most other spheres where high stakes are involved. The WSJ just published “Mistakes in Scientific Studies Surge” (Gautum Naik, 10 Aug 2011) saying “the number of papers published in research journals has risen 44%, the number retracted has leapt more than 15-fold.” Competing explanations: “software has made it easier to uncover plagiarism” and “a more competitive landscape, both for the growing numbers of working scientific researchers who want to publish to advance their careers, and for research journals themselves.” “A single paper in” his journal, one editor said, and “you get your chair and you get your money. It’s your passport to success.” So bet on #2, especially since plagiarism is not the only cause of error.
In public education, one should expect that cheating will continue. As Robert A. Shaeffer, public education director of the National Center for Fair & Open Testing (FairTest), pointed out in the Atlanta Journal Constitution (July 11, 2011), Atlanta was not an isolated case:
“In the past few months, improper test score manipulation have been uncovered in Washington, D.C., Baltimore, Philadelphia, New York City, Los Angeles, Miami, Orlando and many smaller communities. . . . Unreasonable score gain targets coupled with unreasonable pressure on educators is standard operating procedure for testing programs across the nation. Independent analysts conclude that nearly all public schools ultimately will be declared “failing” under the No Child Left Behind mandate of 100 percent proficiency. Teachers and principals face stern sanctions, including job loss, if they do not boost scores.”
Going back a decade, we should also add Houston, winner of the first Broad foundation prize in urban education, where low-scoring students were kept from taking exams, either by being placed in special education classes, given language exemptions or being held back a year. (See Julian Vasquez-Heilig and Linda Darling-Hammond, “Accountability Texas-Style: The Progress and Learning of Urban Minority Students in a High-Stakes Testing Context,” Educational Evaluation and Policy Analysis, v30 n2 p75-110 Jun 2008) While there are lots of such cases, this way of getting around accountability policy incentives is particularly important because Houston’s superintendent, Rod Paige, became Secretary of Education and oversaw the implementation of the NCLB accountability regime.
So here is the real question: What are we doing with testing?
–Some kids are denied recess so they can get more test prep while physical education is taken as a joke. (Among many, see Anne Marie Chaker, Rethinking Recess: As More Schools Trim Breaks, New Research Points to Value Of Unstructured Playtime, WSJ, 10 Oct 2006 and Peter Schworm, “Schools let phys ed slip off schedule,” Boston Globe, 18 Oct 2004; also look at susanohanian.org and do some searches)
We end up not only ignoring the consensus among child development experts (who recommend ‘play, play, play and more play’), but with unhealthy, overweight kids
–The curriculum, especially as it is driven by NCLB’s focus on short term improvements in Reading and Math test scores, is getting narrower and narrower. Phys ed can be lumped in with Art and Music as areas of neglect. But even traditional academic subjects, such as Science and Social Studies get less attention. As for English, forget about Literature, just concentrate on test skills. Overall, what makes school engaging, interesting and fun has been systematically removed.
–Most strikingly, research by Kyung Hee Kim (William & Mary) indicates creativity in US youth has gone down over the last 20 years, just the period that standardized testing and data gathering became the driving force of reform. That is just one researcher and I’m sure there will be others lined up to say differently, but there are extreme dangers in having a data-driven (rather than data-informed) model, especially when non-educators using business-based models become the decision makers.
This is especially true when we consider Joel Klein and Michelle Rhee. First, Ms. Rhee, who wants to raise 1 billion for her StudentsFirst advocacy group, the first contributions coming from Eli Broad. She has supported Gov. Walker in Wisconsin, Gov. Christie in NJ and is an adviser to Rick Scott a Florida Tea Party favorite. It is suggested this is part of and effort to “tie education reform to a larger conservative agenda to crush organized labor.” (Suzy Khimm, “As Michelle Rhee Links Arms With The Right, Allies Worry,” Washington City Paper, June 24, 2011) Is that how she’ll get her funding? I don’t know. I was a DC parent and I know she did some positive things there, especially as regards the school’s operations (supplies, maintenance, etc), and said the right things (if not always doing them) about Early Childhood Ed and Special ed issues. Nonetheless, her educational policies seemed based on the single metric of student achievement as measured by standardized tests. That sort of pressure is not what I want for my child.
Where does that metric lead? Right to Mr. Klein, who left the NYC DOE to be executive vice president at Murdoch’s News Corp. You might have seen him sitting behind Rupert, next to Wendy Murdoch at the Parliament hearings –he’s heading up the company’s internal response to the phone-hacking scandal. But he signed up to News Corp sell educational technology products to school districts, including the one he used to head.
As to how this got by the NYC conflict of interest statutes, I cannot even fathom a guess, but News Corp owns “Wireless Generation, an education technology firm that had worked closely with Klein during his tenure as chancellor on two projects: ARIS, a controversial (and buggy) data system that warehouses students’ standardized test scores and demographic profiles; and School of One, a more radical attempt to use technology to personalize instruction, reorganize classrooms, and reduce the size of the teaching force.” (See Dana Goldstein, Scrutiny on Murdoch’s School Reform Agenda Grows, The Nation, July 21, 2011) Murdoch’s stated belief is that with the “aid of technology, schools could use only the finest teachers in every course, in every subject, at every grade — and make them available to every child. . . . You can get by with half as many teachers.” (Quoted in The Times (London), June 22, 2011) Or maybe even a school with just one.
Thanks,
Brian Ford