An earlier post this morning offered advice about how to read reports about charter school data. A commenter complained that the data in the post specifically referring to Eva Moskowitz’s Success Academy was incomplete and therefore misleading.I asked the author of the post, who works at the headquarters of the Néw York City Department of Education, to respond. The author worked at Tweed during the Bloomberg era.

Here is the response:

“Success Academy’s Numbers

******
“Let’s start by saying that all analysis of Success Academy is difficult because they refuse to be transparent with their data. When the New York State Comptroller attempted to audit Success Academy’s use of public money, Success Academy sued to prevent the audit. And Success Academy is, believe it or not, even less transparent with the data that would answer the questions in How to Analyze False Claims about Charter Schools. Every citizen should encourage Success Academy to openly share their data with the New York City Independent Budget Office or the New York City Comptroller’s Office so that a full evaluation of their numbers can be done.

“What do we know? We know that other analyses have found a similar result as the one shared in the essay. Very few of the Success Academy schools have been around long enough to establish a record. The ones that have show very, very, very high attrition rates.

“One such analysis found “at Success, the pattern is similar, if not more stark. Not only do its classes contain disproportionately few students with disabilities and English language learners (ELLs), but their numbers almost invariably decrease with each passing year. This should have no uncertain effect on test scores. Clearly, the ranks of students with disabilities consistently dwindle. The pattern for students learning English is less consistent but equally egregious. In the first two years of available data, there were hardly any ELLs. In 2010 Success suddenly came up with a nearly representative portion of these students, but their numbers more than halved by the next year. (2012-13 data isn’t yet disaggregated by student demographic.)”

“Insideschools reported that “according to figures on the school’s New York State Report Card, 83 students entered kindergarten in 2006-07, the school’s first year of operation. When that class reached 4th grade in 2010-11, it had only 53 students — a drop of 36 percent. Harlem Success also took in a 1st grade class with 73 students in 2006. When that group reached 5th grade, it too had shrunk appreciably — by 36 percent. The attrition accelerated as the classes advanced. The 2006-07 1st grade class, for example, did not shrink at all as it entered 2nd grade, but saw one sharp falloff between 2nd and 3rd and another between 4th and 5th.”

Yet another analysis found something similar “So the next thing I looked at was their student attrition. If they ‘lost’ many students, these scores are tainted. Now there is only one Success school that has been around since 2007. That school started with 83 kindergarteners and 73 first graders. Those cohorts just tested in 6th and 7th grade, respectively. The school has ‘lost’ a big chunk of those original 156 kids. Of those 73 first graders in 2007, only 35 took the seventh grade test. Of the 83 kindergarteners, only 47 took the sixth grade test last spring. Overall, they have ‘lost’ 47% of the original two cohorts. If this is one of the costs of having such high test scores, I’m not sure if it is worth it.”

“Success Academy rather uniquely tends to open elementary schools that only serve grades K-3 or K-4. This suggests that their attrition rate is high enough that it becomes necessary to combine multiple feeder elementary schools into a single middle school by 5th (or even 4th grade). It has been noted “it may be significant that the bulk of the attrition at Harlem Success Academy 1 seems to have come in the tested grades.”

“The essay analyzed the attrition rate at Success Academy using a different data set, namely the –testing cohort data. This may do a better job of accounting for Success Academy’s approach of holding many students back a grade level which creates a 3rd grade bulge as those students don’t move on to 4th grade. As clearly stated in the essay this method assumes that the size of each entering class is relatively stable from year to year, as they tend to be in the established Success Academy schools. The results are similar to those of other approaches which find attrition rates approaching or exceeding 50% by the end of middle school.

“Success Academy, more specifically Eva Moscowitz, is at a crossroads—they can choose to cancel school, to protest, to walk over bridges, to travel to Albany, to buy TV ads. Or they can choose to be transparent and open and conduct an honest conversation about education, equity and access for all children.”