Indiana has one of the nation’s largest voucher programs (34,000 students) and has been in operation long enough to collect four years of data.
The latest voucher study reached the conclusion that students who get vouchers fall behind in the first two years, notably in math, but eventually catch up to their public school peers.
But do they?
Steve Hinnefeld reported on the release of the study, but then had second thoughts.
He writes:
“The study, by Joe Waddington of the University of Kentucky and Mark Berends of the University of Notre Dame, was released Monday. Its findings were covered by National Public Radio, Chalkbeat, Education Week, the Indianapolis Star and the Fort Wayne Journal-Gazette. A headline in the Washington Post was typical: “School voucher recipients lose ground at first, then catch up to peers, studies find.”
“But the students who “catch up” are only a handful among voucher students included in the study. The study analyzed test scores for 3,913 students who received vouchers during the first four years of the Indiana program, from 2011-12 to 2014-15. But they had four years of test-score data for only about 5 percent of those students.
“The only students to produce four years of data were those who were in certain grades and received a voucher in the first year of the program. (Indiana gives standardized tests to students in grades 3-8, so those who were in sixth grade or higher in the first year of the program would have aged out of the annual tests in four years).
“By contrast, the researchers analyzed two or more years of data for over half the students in the study. So the finding that voucher students lost ground in the first and second year after moving to a private school seems considerably more solid.
“The study also includes data for 15 percent of voucher students who left their private school and returned to a public school. Those students, on average, had fallen further behind in math than the typical voucher student, and they also had lost ground in English/language arts.
“Here’s a possible interpretation: Maybe the voucher students who stayed in private schools for four years weren’t representative of all low-income students who received vouchers. Maybe they were disproportionately high-achieving or ambitious students who were more likely to make a go of it in private school.
“And the students who gave up or lost their vouchers and returned to public schools – maybe they left because their parents saw the private school wasn’t helping them and may have been hurting. Or maybe the private schools, which can set their enrollment standards, gave some of those students a nudge.
“Maybe the message isn’t that voucher students who stick with private schools do OK academically, but that voucher students who do OK academically are more likely to stick with private schools.
“If that’s the case, you can’t make a credible claim that voucher students will necessarily regain what they lost if they just persist in their private school. Some will, but others won’t.”
Thus, when you see a headline saying that voucher students lose ground but eventually catch up to their peers in public school, take a second look. Who persisted long enough to “catch up”?
Weren’t voucher schools supposed to “save poor kids from failing public schools”? There is no evidence for that assertion. If the kids are lucky, or work hard, they just might catch up to their peers in public schools.
Diane Ravitch's blog 
It is shocking to me how often attrition rates are completely ignored as irrelevant in these types of studies.
If this were a drug study comparing the effects of a new treatment protocol against a control group, and the group testing the new treatment was allowed to shed patients with no oversight, the researchers would be censored and drummed out of academia for insisting that the number of missing patients was irrelevant to their conclusions.
But in the ed reform scholarly community researchers find there is a lot of funding to be had for these kind of studies that ignore attrition to claim success for non public schools.
I’m shocked anyone finally noticed. I’m sure this will be the exception rather than the rule as it is much more advantageous for writers to accept results without question.
Sometimes even drug studies, sponsored by the pharmaceutical companies are guilty of cherry picking results or their “findings” are based on a small sample. When research is conducted by a neutral party, it is less likely to be tainted by those with marketplace agendas. The media also repeats these “findings” without due diligence or much scrutiny. The media helps feed the propaganda machine.
Exactly relevant and most disturbing that, of all disciplines, this phenomenon HAS occurred repeatedly in Education: If any study “was allowed to shed patients with no oversight, the researchers would be censored and drummed out of academia for insisting that the number of missing patients was irrelevant to their conclusions.”
The “ed reform scholarly community” – I love it!
To anyone that still supports this false dichotomy of alleged (and confusing) CHOICE between community-based, democratic, transparent, non-profit, traditional public schools and the autocratic, opaque (secretive), corporate and/or voucher-funded private schools, no argument is valid until those so-called cherry picking CHOICE schools are as transparent in information and open to every student as the traditional public schools they are robbing public money from.
As long as those alleged CHOICE schools control the information flow of their performance, there is no way to determine the validity of any claims.
YES, Lloyd: “As long as those alleged CHOICE schools control the information flow of their performance, there is no way to determine the validity of any claims.”
Love your comments.
When I saw that study I had this gut feeling that something was just not kosher. Now I know what it is – all those voucher students who were ‘counseled’ out. This is just as bad as 100% of the graduates going to college and never mention how many were pushed out before graduation.