The media loves the story of miracle schools. Imagine that! A school where 90% or more pass the state tests! Where 100% graduate. Where 100% are accepted into four-year colleges. Michael Klonsky once said to me, miracles happen only in the Bible. When the subject is schools, miracle claims should be carefully investigated.

With that caution and skepticism in mind, we turn again to a post by a researcher who works for the New York City Department of Education and must remain anonymous. This is the same researcher who chastised the media for ignoring attrition rates at Eva Moskowitz’s Success Academy schools. In posting that article, I failed to capture the links to documentation (a terrible oversight, I admit). I include his/her links at the bottom of this article.

Ed Reformers Are Most Like (a) Pinocchio (b) Beavis:
Getting to the Bottom of the Reformer Distaste for Honest Analysis

My short essay examining some of the dishonest claims about Success Academy’s data led to interesting debate on this blog.[1] Some of that discussion illuminated the dishonesty with which education reformers approach data and facts. I’ll limit this essay to the dishonesty reformers display in the charter school debate.

Reformers tend to make two very different arguments about charter schools. Argument #1 is that charter schools serve the same students as public schools and manage to put public schools to shame by producing amazingly better results on standardized exams. Therefore, reformers claim, if only public schools did what charter schools do (or better yet, if all public schools were closed and charter schools took over), student learning would dramatically increase and America might even beat South Korea or Finland on international standardized tests. When it is pointed out that, as a whole, charters do no better than public schools on standardized tests [2], reformers will quickly turn their attention to specific charter chains that, they claim, do indeed produce much better standardized test results. So what’s the deal with these chains? Well, in every case that has been subjected to scrutiny their results are extremely suspicious. Here is a short list of examples:

1. Achievement First in New Haven had a freshman class of 64 students (2 students enrolled later), and only 25 graduated- a 38% graduation rate- yet the school claimed a 100% graduation rate by ignoring the 62% attrition rate. [3]

2. Denver School of Science and Technology (DSST) had a freshman class of 144 students and only 89 12th graders- a 62% graduation rate- yet the school (and Arne Duncan) claimed a 100% graduation rate by ignoring the 38% attrition rate. [4] As a 6-12 charter chain, DSST also manages to attrite vast numbers of their middle school students before they even enter the high school.

3. Uncommon Schools in Newark disappears 38% of its general test takers from 6th to 8th grade.[5] Another analysis found that through high school the attrition rate was, alarmingly, much higher “Uncommon loses 62 to 69% of all males and up to 74% of Black males.”[6]

4. BASIS in Arizona- “At…BASIS charter school in Tucson, the class of 2012 had 97 students when they were 6th graders. By the time those students were seniors, their numbers had dwindled to 33, a drop of 66%. At BASIS Scottsdale…its class of 2012 fell from 53 in the 6th grade to 19 in its senior year, a drop of 64%.” [7]

5. The Noble Network in Chicago- “Every year, the graduating class of Noble Charter schools matriculates with around 30 percent fewer students than they started with in their freshman year.” [8]

6. Harmony Charters in Texas- “Strikingly, Harmony lost more than 40% of 6th grade students over a two-year time.” [9]

7. KIPP in San Francisco- “A 2008 study of the (then-existing) Bay Area KIPP schools by SRI International showed a 60% attrition rate…the students who left were overwhelmingly the lower achievers.” [10]

8. KIPP in Tennessee had 18% attrition in a single year! “In fact, the only schools that have net losses of 10 to 33 percent are charter schools.” [11]

In every case these charter chains accepted students that were significantly more advantaged than the typical student in the district, and then the charters attrited a significant chunk of those students.

Success Academy in New York City plays the same game. It accepts many fewer high needs special education students, English Language Learners, and poor students. [12] It attrites up to 1/3 of its students before they even get to testing grades and then loses students at an even faster pace. It selectively attrites those students most likely to get low scores on standardized tests. [13] It is legally permitted to mark its own exams (as are all New York City charter schools) while public schools cannot. It loses 74% of its teachers in a single year at some of its schools. [14] The author of the Daily News editorial that sparked the initial blog commented “even in the aggregate that wouldn’t seem to account for” the results. It is entirely unclear what he means by “in the aggregate.” But it is clear that he has his arithmetic wrong. A charter chain that starts with an entering class that is likely to score well on standardized tests, then selectively prunes 50% or more of the students who don’t score well on standardized tests and refuses to replace the disappeared students with others, can easily show good standardized test results with the remaining students. Any school could do this. It’s really not rocket science.

Charter advocates usually first give argument #1 a try. When called on the data that clearly show high-flying charters engage in creaming and in pruning, which can account for most of their “success,” they quickly switch to argument #2. Argument #2 claims that charter schools play a different role than public schools. What exactly their role is can vary from “serving high-potential low-income students [14]” to serving as laboratories of innovation. The problem with argument #2 is that we don’t need charters to cream students (public schools could do that too…if it were legal), and charters as a sector are not doing anything innovative. Kicking out half of your class is no innovation, nor is it hard to create an environment that will encourage the half least likely to succeed to quit. The Navy SEALs have been doing that for years.

At the policy level these two different arguments have led to much confusion. It is often unclear what charter advocates are defending as they switch back and forth between the two arguments. This makes it difficult to have sensible public discussion about charters and leads many to accuse charter advocates of hiding their true motivations (from privatizing education for profit to breaking unions).

It is time that education policy makers demanded an honest accounting of charter practices. Metrics must be produced by every district clearly showing the demographics of charter school students, the attrition rate, and general data on which students are attrited. It is critical that the demographic data be as detailed as possible (e.g. specifying level of special education need, distinguishing between free and reduced price lunch, specifying level of English Language Learner status) since the charter sector and its advocates have in the past used broad categories to cover up important differences (e.g. claiming to serve the same numbers of English Language Learners as public schools while only serving advanced ELLs, claiming to serve the same number of poor students as public schools while serving much higher proportions of reduced as opposed to free lunch students, claiming to serve the same number of special needs students as public schools while serving only students with minimal needs).[15] With honest data in hand, the more important conversation about good teaching practices, engaging curricula, and effective students support services can begin. It is this conversation that will truly improve education for students. It is also the conversation that professional educators want to have.[16]

[1] https://dianeravitch.net/2014/08/22/is-eva-moskowitz-the-lance-armstrong-of-education/
[2] http://www.washingtonpost.com/blogs/answer-sheet/wp/2013/09/24/the-bottom-line-on-charter-school-studies/
[3] http://jonathanpelto.com/2013/05/30/another-big-lie-from-achievement-first-100-percent-college-acceptance-rate/
[4] http://garyrubinstein.wordpress.com/2014/04/16/arne-debunkin/
[5] http://schoolfinance101.wordpress.com/2010/12/02/truly-uncommon-in-newark /
[6] http://danley.rutgers.edu/2014/08/11/guest-post-where-will-all-the-boys-go/
[7] http://blogforarizona.net/basis-charters-education-model-success-by-attrition/
[8] http://jerseyjazzman.blogspot.com/2012/04/no-bull-in-chicago.html
[9] http://fullerlook.wordpress.com/2012/08/23/tx_ms_charter_study/
[10] http://parentsacrossamerica.org/high-kipp-attrition-must-be-part-of-san-francisco-discussion/
[11] http://www.wsmv.com/story/22277105/charter-schools-losing-struggling-students-to-zoned-schools
[12] https://dianeravitch.net/2014/03/12/fact-checking-evas-claims-on-national-television/
[13] https://dianeravitch.net/2014/02/28/a-note-about-success-academys-data/. The high attrition rate before testing in 3rd grade may explain the data pattern noted in this http://shankerblog.org/?p=10346#more-10346 analysis.
[14] http://www.citylimits.org/news/articles/5156/why-charter-schools-have-high-teacher-turnover#.U_gqR__wtMv
[15] http://edexcellence.net/commentary/education-gadfly-daily/flypaper/2013/the-charter-expulsion-flap-who-speaks-for-the-strivers.html
[16] http://schoolfinance101.wordpress.com/2012/12/03/when-dummy-variables-arent-smart-enough-more-comments-on-the-nj-credo-study/ I leave it as an open challenge to Ms. Moskowitz to voluntarily share this date (scrubbed of identifying student information of course) so that independent researchers can examine the Success Academy results. If she declines to do so we can only wonder what she is hiding.
[17] I wanted to end on a positive note so I add this comment as a footnote. We can expect that reformers will resist allowing the national conversation to go in this direction since they have so little to contribute to it. So many have so little classroom experience and so little time in schools that they will do all they can to make sure the conversation does not turn in this direction. If it did, they’d be out of a job. So we can expect that, as long as reformers maintain their power base, the national conversation about education will be limited to accountability, choice, standards, VAMs… anything but discussion of actual classroom and school-level practices.