An insider in the New York City Department of Education was disturbed to read the New York Times’ editorial praising the CREDO study of charters in New York City. She knew that the data on the public website of the Department of Education does not support the CREDO analysis.

Here is her own analysis, based on DOE’s own data:

A New York Times editorial Saturday praised a new study claiming that through the 2010-11 school year New York City charter schools have produced better results for students than other public schools. Of course this study did not mention the investigative reporting by Reuters proving that charter schools have truckloads of schemes to turn away and kick out students who might bring down their numbers. See that story here.

Nonetheless we took the report on good faith and attempted to verify its claims using the New York City Department of Education’s own data sets that can be found here: http://schools.nyc.gov/Accountability/tools/report/default.htm In these data sets charter school outcomes are compared to those of similar schools. Similar schools are schools that educate comparable students based on incoming test scores and other criteria. We decided to spend some time examining these data sets for the 2009-10 and 2010-11 school years. There are the years in which the NY Times claimed charter schools had “whopping” results. But is it true? Sadly the NY Times has been fooled.

Let’s start with elementary and middle charter schools during the 2009-10 school year. In that year the average student results on the English exam for charter schools in New York City placed them at the 32.5th percentile of similar schools. Looking at students who scored as “proficient” on the English exam charter schools were at the 31.4th percentile of similar schools. And when looking at how charter schools helped their students improve as compared to prior years (in other words: student growth) they performed dismally, ranking at the 20.1st percentile.

But how about math? Maybe charters don’t do such a good job with English but do a better job in math. Well they do, but still performed worse than about 60% of public schools with similar student populations: ranking at the 42.3rd percentile for average student test scores, the 37.9th percentile for students scoring “proficient” on math, and coming in at the 44.8th percentile for student improvement in math.

In 2010-11, the school year in which the study cited by the New York Times claimed the best performance by charter schools, the truth is that charter schools continued to do a much, much poorer job for students than other schools with similar students. In English they ranked at the 35.1st percentile for average student test scores, at the 36.7th percentile for students scoring “proficient” and at the lowly 28.6th percentile for student growth. Math was slightly better: 51.5th percentile for average student scores, 55.4th for student “proficiency,” and 52.2nd for student growth. And this is with the charter school practice of kicking out students right before testing time as shown in this expose http://www.edwize.org/middle-school-charters-show-alarming-student-attrition

So where does this leave us? Charter schools, in fact, did much, much worse than similar schools in NYC in English and about average in math. If you average the numbers for 2010-11 together, charter schools are doing 16.5% worse than the average similar school in English. And, using the same calculation, they do 3% better than the average similar school in Math. It would appear that much, much poorer performance in English and barely better performance in Math does not support the claim that charter schools in New York City give superior results, whatever the New York Times might say. It should also be pointed out that these comparisons are based on ACTUAL students unlike the study cited so approvingly by the New York Times, which invented virtual students to do their “comparisons.”

How about high schools? Maybe charter high schools in New York City are doing a “whopping” job there. Again, unfortunately for students, they do not. Using the data set available on the New York City Department of Education webpage here: http://schools.nyc.gov/NR/rdonlyres/C8903442-BA48-4248-B4CE-156FAA2D8929/0/2010_2011_HS_PR_Results_2012_03_16.xlsx we find that in 2010-11, for the first time, schools were rated based on how well they were preparing students for college. And how did charter high schools do on those ratings? Only 14.9 percent of the charter high school students met the college ready standard as compared to a 32.3 percent average in similar high schools. Only 31.4 percent of charter high school students took and passed a college preparatory course as compared to a 42.8 percent average in similar high schools. And only 52.8 percent of students enrolled in college as compared to a 61.2 percent average in similar schools.

What does this all mean for education? We must start to evaluate the success and failures of initiatives such as charter schools truthfully without letting politics get in the way. We owe this to students. Unfortunately, the response from charter schools, their protectors and funders will probably be a redoubled effort to screen and selectively prune students at charter schools to make their numbers look better. They will continue, with the support of the editorial page of the New York Times, to bash public schools. Instead of committing to improving education for all students and giving all schools the resources to do what we know helps all kids (strong curriculum, small classes, a pleasant school environment, high quality after-school programs, embedded systems of social-emotional and health supports) they will continue to play politics with public education and the futures of our children.