Christopher Lubienski is a professor of educational policy, organization, and leadership at the College of Education at the University of Illinois. He has written extensively about markets and schools. His most recent book is “The Public School Advantage: Why Public Schools Outperform Private Schools.”

He writes here about the recent studies of vouchers:

For years we’ve heard about how the most rigorous studies of voucher programs consistently show significant gains for students — especially urban minority students — and no evidence of harm.  While that claim was highly questionable, it was nonetheless a central talking point from voucher advocates intent on proving that vouchers boost academic achievement.  The idea that vouchers didn’t hurt, and probably helped, the students trapped in failing urban schools and most in need of options was used to justify calls for the expansion of vouchers from smaller, city-level policies to state-wide programs open to an increasing number of students.

Now, a slew of new studies and reviews — including some conducted by the same voucher advocates that had previously found vouchers “do no harm” — is telling quite a different story.  New reviews of existing voucher studies are pointing out that, overall, the impact on the test scores for students using vouchers are sporadic, inconsistent, and generally have “an effect on achievement that is statistically indistinguishable from zero.”

But some new studies on vouchers in Louisiana raise substantial concerns, finding that students using vouchers were significantly injured by using vouchers to attend private schools.

First, kudos to some of the study authors who have previously identified themselves as voucher advocates, in that they had the integrity to publish their findings.

But, what is particularly interesting here is the apparent confusion on the part of voucher-oriented reformers over these new results.  After all, they have a strong theoretical account of how using vouchers to enroll in private schools will lead to greater gains in student learning.  The fact that the new evidence shows otherwise is disorienting for reformers who had believed that private schools are better, and that moving poor students from public schools to private schools would lead to better outcomes.  After all, according to them, at least, all the previous research supported their theory.

So what happened?  Voucher proponents have wondered if the program was too new, too big, or open to too many private schools that had little experience with poorer children.

Actually, perhaps the past results were not so clear, and the new findings were not so unpredictable.

To understand why, let’s consider a related, even over-arching question before we return to these new voucher studies.  Vouchers are premised on the assumption that private schools are more effective.  It’s not just a matter of students in private schools getting higher scores on standardized tests, since it’s well known that they tend to serve, on average, more affluent students who would likely score higher no matter which type of school they attended.  Indeed, some early studies on the public-private question from the 1980s and 90s indicated that, even when researchers control for student demographics and affluence, private, and especially Catholic schools in particular, appear to boost student performance more than do public schools — the so-called “private school effect.”

However, more recent research, including some I have collaborated on, has been showing the opposite: that public schools are actually performing at a level equal to or above private schools, and are thus often more effective.  Such findings turn the assumptions for vouchers upside-down.  Why would we want to move students from public schools, which the current generation of research demonstrates are actually more effective on average, to a less effective educational experience in private schools?

Voucher advocates thought they had an answer to that.  They argued that their favored reports were “gold standard” studies that showed that private schools are better.  Except those studies didn’t show that.  Even if we accept their findings at face value, what they actually indicated is only that in a relatively few cases the types of students who would use a voucher to leave a failing urban public school for a presumably better private school sometimes scored higher.  That is hardly grounds for scaling up the programs to broader populations.  Those voucher studies are not drawing on representative samples of students, or representative samples of public or private schools, and thus tell us nothing about the question of what types of schools are more effective.  Nevertheless, voucher advocates made this claim in an attempt to show their preferred reform leads to higher scores, and that we should thus expand these programs.

In fact, there is much evidence to suggest that improvements in student learning may have less to do with the type of school, but instead depend largely on the types of students served by a school.  That is, vouchers sort students into more academically inclined groups by sending successful applicants to private schools, where they enjoy a beneficial “peer effect” of more affluent classmates, but not necessarily better teaching.  The problem is that there are only so many affluent or academically inclined peers to go around, and as voucher programs are scaled-up, that beneficial peer effect dissipates.

Now, with the new results being released for a larger program in Louisiana, this may be exactly what we’re seeing.  Students using vouchers in this state-wide program enrolled in poorer private schools with fewer affluent and academically inclined peers available to improve the learning experience.

These new findings are a direct contradiction to the frequent claim that no student has been shown to be harmed by vouchers.  This raises an ethical issue.  If vouchers have been an “experiment,” and randomized trials modeled after medical trials are showing evidence of harm, should policymakers end the experiment, as would be the case in medical research?

Regardless, now that voucher advocates are facing evidence challenging their claim that vouchers for private schools boost student performance, expect to see a further retreat from the test score measures they had been embracing to promote their claim on private school effectiveness.  Instead, voucher proponents will move the goalposts and ask us to pay attention to other measures, such as persistence or parental satisfaction, instead.  The problem is, those alternative measures are also problematic and susceptible to peer effects, as time will tell.