A data analyst who worked for the past several years in the New York City Department of Education wrote the following about Eva Moskowitz’s Success Academy charter chain. Most of the data he cites comes from public records maintained by the city or state education departments. His footnotes are at the bottom of the post.

Building a Charter Chain, and a Mayoral Campaign, on Lies

Over the past few days a deluge of, what for lack of a better word can only be described as Success Academy propaganda material, has appeared in the New York City media. The New York Daily News published an opinion piece written by what they termed “a researcher” and a “graduate student” that used the veneer of data to argue that Success Academy is a true success and miracle story.[1] Careful analysis reveals that many of the claims are outrights lies and the rest are half-truths.

Let’s start digging in.

Claim #1: “Success Academy schools serve a similar share of special needs students relative to their zoned counterparts.” This is a lie. According to the latest public data on the New York City Department of Education’s Progress Report website [2] the 4 Success Academy schools in Harlem had a total of 2540 students enrolled. Of these students only 17 were special education students with the highest level of need. That’s 0.6% of their students. By contrast the average percent of special education students with the highest level of need at community elementary/middle schools throughout New York City was 9.4% and 14% in Harlem. The average community school in NYC serves 1,500% MORE of the highest need special education students than Success Academy.[3] Such a vast disparity, in what to a significant extent are disabilities based on neurological, medical and physical differences, can’t be explained by anything other than Success Academy not serving the neediest students in the community.[4] You might think it would be hard for the researchers who penned the propaganda in the Daily News to top this lie. They will manage to surprise you.

Claim #2: “the school is more successful in teaching students English…27% of Success Academy students passed the ELL writing exam, compared to 19% at nearby schools.” This is a lie. The school is not successful at teaching English Language Learners. The “researchers” somehow forgot to mention that Success Academy only serves ELLs who are already amazingly good at English. According to the New York State data [5] over 90% of the ELL students at Success Academy are proficient or advanced in kindergarten (that is before Success Academy would have had much of an effect). Seems that Success Academy only serves English Language Learners who already know English. There ARE data suggesting that Success Academy IS very successful at kicking ELLs out (perhaps the few intermediate level ELL students who manage to slip through the lottery).[6]

Claim #3: “Success Academy students scored on average 39 points… ahead of others from equivalent backgrounds.” This is a lie. It is only by misrepresenting the “equivalent background” that they can make this claim. As we have seen in the first two lies that were exposed above, these “researchers” have not even come close to controlling for “equivalent backgrounds.” They have not controlled for level of special education need. They have not controlled for English Language Learner performance levels. They have not controlled for parent characteristics. They have not controlled for home environment. They have not controlled for peer effects of creaming some of the most advantaged students in each neighborhood.[7]

Claim #4: “the overall rate of attrition at Success Academy is far from alarming.” This is a lie. The “researchers” make this claim by comparing the yearly rate of attrition at Success Academy to that of other, nearby community schools. But that is an absurd comparison to make. After all, the data show that Success Academy is serving some of the most economically, linguistically, and academically privileged students in Harlem.[8] On top of that parents must actively choose to enter a lottery to get into Success Academy and must put up with Ms. Moskowitz’s hazing.[9] The fact that even after such a thorough screening process every single year, year after year, another 10% of the student body leaves IS rather disturbing.

Claim #5: They dismiss as a “side argument” the notion that “the choice not to backfill drives up scores.” It is a moral imperative to point out that only charter schools are given this “choice.” Public schools serve all students, from all circumstances, at all times (even when charter schools kick those very students out right after “census day,” the day districts use to calculate enrollment for budget purposes). It is disturbing that this difference seems to carry such little weight with so-called education reformers. It can only make one wonder how invested they really are in the success of each and every student.

It is also disturbing that these researchers seem unable to use basic logic and arithmetic. An Independent Budget Office report showed that charters are more likely to lose the students who score poorly on the New York State exams and who are more often absent.[10] Losing 30% of the students who will test poorly before they enter the first testing grade (i.e. 10% attrition each year through 3rd grade) can have a huge effect on test outcomes. Since it is reasonable to assume that Success Academy replaces those students with ones more likely to do well on the exams, the whole Success Academy effect can be explained by attrition.[11]

Let’s use the data from Harlem Success Academy 1 as an illustration. This year’s 8th grade cohort, the one that started kindergarten in the 2006-07 school year, dropped from 83 students in kindergarten to 63 students in 3rd grade. This means that at least 25% of the cohort disappeared even before the first exam.[12] As the disappearing students are the ones least likely to do well on the exams that means that in 3rd grade these students can be expected to score proficient on the New York State exams at a rate that would be about 25% higher than would be expected based solely on other factors (such as the creaming and self-selection noted above). This effect fully accounts for Success Academy’s, now obviously banal, outcomes.

Claim #6: “growth data from 2013 suggests that in the upper grades, on average, students maintain their high early achievement rather than moving further ahead.” Now this one is interesting since the researchers are more or less admitting that a Success Academy education amounts to very little. If students are not growing more the more time they are spending in these schools, what exactly is Success Academy accomplishing? This little fact shows that the rest of their essay amounts to little more than apologetics and lies. It is implausible to assume that Success Academy accomplishes magic in the early grades and then barely manages to hold ground in upper grades. In fact, this is another piece of evidence suggesting that Success Academy amounts to little more than an accounting trick.

However even this is a half-truth since the most recent (2012-13) New York City Department of Education Progress Report data show that Success Academy lagged well behind its “peer” schools in English exam growth.[13] Success Academy scored in the 39th percentile on English exam growth for their overall student population and in the 21st percentile on English exam growth for the students who began with scores in the lowest 1/3 of students citywide.

Claim #7: “The implication is that, through “drill and kill” instructional techniques, Success Academy is teaching students only low-level skills…The reality is much different.” Here the “researchers” mislead in various ways. They state that “Success Academy students get more science instruction than their peers.” It is unclear how they know this since no evidence is cited. They forgot to mention that according to a Success Academy teacher “We do not teach history or foreign languages in elementary school.[14]” The teacher also revealed that “Test prep starts in November.” A former teacher noted that the “Entire school focused on remaining at top of network schools assessment wise.” [15] From another interview, “All of the other grades, besides seventh and eighth grade, have been doing test prep since…the beginning of November. So that means they weren’t having English class, they were just doing stupid passages by random authors of no literary basis, quality, and just doing multiple choice questions for the past two months or so. [16]”

The “researchers” also somehow forgot to mention that exactly zero Success Academy students scored well enough on the Specialized High School Admission Exam to be admitted to one the city’s top high schools.[17] This forgotten piece of data supports the teachers’ claims that a very narrow sort of test prep characterizes Success Academy curriculum, as little of it appears to generalize to other exams.

Conclusion: Given all the other tricks in the Success Academy playbook including suspension rates 2-3 times the district averages [18], teacher attrition rates approaching 75% a year [19], the ability to spend thousands of more dollars per student thanks to deep pocketed ideological foes of public schools [20], the ability to grade its own exams (public schools are legally prohibited from doing so), and lots more instructional time, Success Academy must be considered an utter failure.

The media, as seen in the “puff piece” in the New York Times Magazine [21], is clearly getting behind Eva Moskowitz’s planned mayoral campaign. However, the accumulating lies cannot withstand scrutiny. Eva has fought audits of her schools and has refused to be transparent with Success Academy data. These are not the characteristics that we want in an elected official. Nor in a leader of schools.

[1] http://www.nydailynews.com/opinion/results-real-article-1.1929656
[2] http://schools.nyc.gov/Accountability/tools/report/default.htm
[3] By the way, Success Academy did not do any miracles with these students. Fewer than 20% tested proficient on the NYS English Exam.
[4] Not that that has stopped shills for the charter school industry from trying http://nepc.colorado.edu/files/ttr-charter-speced-crpe-mead.pdf
[5] http://data.nysed.gov/
[6] http://commonal.tumblr.com/post/58209601458/harlem-success-academy-charter-and-attrition
[7] They also convert a 39 point difference in test scores into being “3 years” more advanced than other students. To put this ridiculous claim in context, let’s note that 39 points is about 12% of the average total score for general education students in New York City on the New York State exams. They are essentially claiming that a student who scores a 92% on an exam is 3 years more advanced than a student who scores an 80%.
[8] http://andreagabor.com/2014/05/09/a-demographic-divide-in-east-harlem-the-neediest-kids-go-to-public-schools-not-charters/
[9] Here is one parent’s description on insideschools.org: “I attended the orientation & was “turned off” by Ms. E. Moskowitz condescending & offensive approach. She was bordering on insulting. When informing the parents of their obligations to attend the after-school games/activities, she said,“All parents are expected to attend and stay for the entire time. Don’t think you can come for a little while & leave to go get your hair done”. “Another comment made in poor taste was when a parent ask if there was financial asst. 4 uniforms. Ms. M’s response was, “No.you have six weeks to save up”.”
[10] http://www.ibo.nyc.ny.us/iboreports/2014attritioncharterpublic.pdf “Among students in charter schools, those who remained…had higher average scale scores…compared with those who had left for another New York City public school.” “Absenteeism is an even greater predictor of turnover for students in charter schools, compared with its predictive power for students in nearby traditional public schools.”
[11] Since Success Academy refuses to transparently share its data we are unable to determine exactly how much of a role attrition plays in the early grades. The public data only show overall cohort sizes, so when Success Academy loses 10% of its students a year, if it backfills those seats in grades K-3, the cohort size appears to remain stable through those years. As the “researchers” admit to at least the 10% annual attrition rate we will accept their numbers. But it is also possible that attrition is even higher in the early grades, since “survivor bias” makes it reasonable to assume that the longer a student has been at a Success Academy school the more likely they are to remain.
[12] See the previous endnote for an explanation of why this is likely an underestimate.
[13] http://schools.nyc.gov/NR/rdonlyres/1550033E-3F15-4746-BD1A-DF3364721785/0/2012_2013_EMS_PR_Results_2014_04_24.xlsx this data does not account for the selective attrition effects noted above and is therefore a very optimistic figure for the real outcomes within Success Academy schools
[14] https://dianeravitch.net/2013/10/04/mole-in-success-academy-speaks/
[15] http://www.glassdoor.com/Reviews/Success-Academy-Charter-Schools-Reviews-E381408_P5.htm
[16] http://honestpracticum.com/exclusive-interview-a-tfa-teacher-working-at-success-academy-part-1/
[17] http://www.nydailynews.com/new-york/education/gonzalez-success-charter-students-fail-top-city-schools-article-1.1833960
[18] http://www.nydailynews.com/new-york/education/success-academy-fire-parents-fight-disciplinary-policy-article-1.1438753
[19] http://www.wnyc.org/story/302768-high-teacher-turnover-at-a-success-network-school/ and http://www.citylimits.org/news/articles/5156/why-charter-schools-have-high-teacher-turnover#.VA60YP_wvcw
[20] http://nepc.colorado.edu/files/ttr-charter-rent_0.pdf
[21] http://www.huffingtonpost.com/diane-ravitch/charter-schools_1_b_5781474.html