A couple of days ago, New York Times’ writer David Leonardt wrote a column endorsing Question 2 in Massachusetts, which would expand charters by 12 a year for the indefinite future. He presented some studies to buttress his view that charters are “schools that work,” which he defines as “high expectations, high support.” He visited the MATCH charter school and talked to some of the researchers, who were excited about their findings. Leonardt acknowledged that not all charter schools were as effective as the ones in Boston, but nonetheless thought it was a good idea to authorize more charters in Massachusetts.

I invited Jersey Jazzman, who is a teachers and also a graduate student at Rutgers, to review the column and the evidence. Here is his response, as he is an expert on charter research.

He begins by suggesting that the comparisons in the study cited by Leonhardt are inadequate.

Because simply showing that charter school students in Boston get better test scores than similar students in the Boston Public Schools is not, I’m afraid, nearly enough evidence to support lifting the cap. In fact, the more time I spend looking at this research, the more questions I have about whether Massachusetts can reasonably expect charter expansion to improve its schools:

– Are the students who enter charter lotteries equivalent to the students who don’t? This is a critical limitation of these studies that is often ignored by those who cite them. The plain fact is that the very act of entering your child into a charter school lottery marks you as different from the rest of the population; you are taking an affirmative step the majority of public school parents are not taking in an attempt to improve your child’s education. There’s a real likelihood your family is not equivalent to a family that doesn’t enter the lottery…

It’s not always clear how to calculate the overall target population in these studies; I used the Ns that made the most sense to me.** But even if we’re not quite sure about the exact numbers, the scope of the issue is clear: the study sample is only a fraction of the total population. Which would be fine — if the sample was randomly drawn from the target population.

But clearly, that’s not the case: The sample is self-selected, because families have to choose to enter the lottery. Which means the results of the study can only be generalized to that population, because there may be characteristics of the students in the sample that are different from the entire Boston population and affect test scores….

First of all, how do we know the charters are any different than the traditional public schools regarding these school practices? Angrist, Pathak & Walters (2012) surveyed charters for their practices, which is fine… except we don’t really know how they compare to the public district schools. If we’re going to ascribe effects to these practices, we should know how they differ across our treatment and control schools.

We can say, however, that the charters have longer school days and years. This is probably a significant contributor to any effects the charters show. But is it necessary to expand charters to lengthen instructional time? Can’t Boston just do that in its district schools?

JJ points out that charters are known for their reliance on inexperienced teachers who burn out and leave within 2-3 years. Does the Boston area have enough wannabe teachers to staff a growing number of charters? Is it really a good idea to rely on policies of churn-and-burn for teachers, continually recycling inexperienced teachers to meet the demand for them?

He says that the question of cost is central to the proposal for expansion. Parents understand that more money for new charters means less money for public schools, their own community’ public schools.

JJ writes:


And, yes, there are costs. As this clever model developed by a couple of public school parents shows, districts can’t easily absorb the costs of charter expansion, which is why the state offers extra funds. Unfortunately, the state has not fully funded this program in recent years; if they can’t find the money now, how will they find even more funding in the future?

We know that charters place fiscal burdens on hosting districts, largely because they educate students who would otherwise go to private school and they replicate administrative and other costs by creating multiple systems of school governance. We know that charters are not held to the same standards of transparency and accountability as public district schools, because they are not state actors. This has created major problems in other states, incentivizing behaviors that are not in the public’s interest.

Is it not possible, given all this, that Boston’s charters are getting good results because of the cap? That limiting their expansion has increased quality and stopped the abuses that have plagued states like Michigan, Ohio, and Florida, which have let charter expansion run wild?

He concludes:

I understand Boston’s children can’t wait any longer for real improvement in their schools and their lives. But lifting the cap largely on the basis of these limited studies is not, in my opinion, smart public policy. The good people of Massachusetts have every right to question whether voting yes on Q2 is in the best interests of students both in and out of charters, and to consider the limits of the evidence presented to them as they make their decision.

And I would add, taking money away from the schools that serve the overwhelming majority of children in Massachusetts so as to open schools for a tiny minority of other students makes no sense from a public policy standpoint–or common sense. Why weaken the public schools that serve more than 90% of children in public schools to benefit the few?

As for David Leonhardt, I suggest that he visit charter schools in Minnesota, Nevada, Ohio, and many other states, where charter schools are among the lowest performing schools in the state. Anecdotes do not make good public policy, nor does one visit to a “no-excuses” charter school in Boston.