Archives for category: Research

On July 13, I posted the abstract from the study referenced here, showing that private schools are not better than public schools when demographic variables are controlled. If you have a school composed of kids from rich and educated families, your school will get higher test scores than a school that is open to all students.

Valerie Strauss has an extended discussion of the study here.She interviewed one of the study’s authors.

University of Virginia researchers who looked at data from more than 1,000 students found that all of the advantages supposedly conferred by private education evaporate when socio-demographic characteristics are factored in. There was also no evidence found to suggest that low-income children or children enrolled in urban schools benefit more from private school enrollment.

The results confirm what earlier research found but are especially important amid a movement to privatize public education — encouraged by Education Secretary Betsy DeVos — based in part on the faulty assumption that public schools are inferior to private ones.

DeVos has called traditional public schools a “dead end” and long supported the expansion of voucher and similar programs that use public money for private and religious school education. According to the National Conference of State Legislatures, 27 states and the District of Columbia have policies allowing public money to be used for private education through school vouchers, scholarship tax credits and education savings grants.

Related: [There is a movement to privatize public education in America. Here’s how far it has gotten.]

The new study was conducted by Robert C. Pianta, dean of U-Va.’s Curry School of Education and a professor of education and psychology, and Arya Ansari, a postdoctoral research associate at U-Va.’s Center for Advanced Study for Teaching and Learning.

“You only need to control for family income and there’s no advantage,” Pianta said in an interview. “So when you first look, without controlling for anything, the kids who go to private schools are far and away outperforming the public school kids. And as soon as you control for family income and parents’ education level, that difference is eliminated completely.”

Kids who come from homes with higher incomes and parental education achievement offer young children — from birth through age 5 — educational resources and stimulation that other children don’t get. These conditions presumably carry on through the school years, Pianta said…

The Pianta-Ansari study examined not only academic achievement, “which has been the sole focus of all evaluations of private schooling reported to date, but also students’ social adjustment, attitudes and motivation, and even risky behavior, all of which one assumes might be associated with private school education, given studies demonstrating schooling effects on such factors.” It said:

“In short, despite the frequent and pronounced arguments in favor of the use of vouchers or other mechanisms to support enrollment in private schools as a solution for vulnerable children and families attending local or neighborhood schools, the present study found no evidence that private schools, net of family background (particularly income), are more effective for promoting student success.”

And it says this:

“In sum, we find no evidence for policies that would support widespread enrollment in private schools, as a group, as a solution for achievement gaps associated with income or race. In most discussions of such gaps and educational opportunities, it is assumed that poor children attend poor quality schools, and that their families, given resources and flexibility, could choose among the existing supply of private schools to select and then enroll their children in a school that is more effective and a better match for their student’s needs. It is not at all clear that this logic holds in the real world of a limited supply of effective schools (both private and public) and the indication that once one accounts for family background, the existing supply of heterogeneous private schools (from which parents select) does not result in a superior education (even for higher income students).”

Pianta and Ansari note in the study that previous research on the impact of school voucher programs “cast doubt on any clear conclusion that private schools are superior in producing student performance.”

Valerie goes on to refer to an important study by Christopher and Sarah Lubienski:

A 2013 book, “The Public School Advantage,” by Christopher A. Lubienski and Sarah Theule Lubienski, describes the results of a look at two huge data sets of student mathematics performance, that found public school students outperform private school ones when adjusted for demographics.

Categories Education Reform, Research, Supporting public schools

North Carolina: NEPC With More About Analysis of That Amateurish, Cooked Voucher Study

By dianeravitch

July 15, 2018 //

I have posted two critiques of the North Carolina voucher study that claimed great gains for students who took vouchers to learn that dinosaurs and humans co-existed.

Here is another, which is probably definitive and all you need to know. It was posted by the National Education Policy Center.

An evaluation of an education program typically gives some information about whether or not a program is working. But a recent evaluation of North Carolina’s school voucher program is so flawed methodologically that it fails to explain whether the state’s Opportunity Scholarships help or harm a student’s education, according to a review by Kris Nordstrom, an education policy consultant on the Education and Law Project at the North Carolina Justice Center, a social justice-focused research and advocacy organization.

Nordstrom’s review is part of a new NEPC feature called Reviews Worth Sharing, which are not commissioned or edited by NEPC but that we believe contribute to our goal of helping policymakers, reporters, and others assess the social science merit of reports and judge their value in guiding policy. The views and conclusions addressed belong entirely to the author.

The evaluation reviewed, An Impact Analysis of North Carolina’s Opportunity Scholarship Program on Student Achievement, is a working paper by North Carolina State researchers Anna J. Egalite, D.T. Stallings, and Stephen R. Porter.

The review finds that methodological flaws in the evaluation make it impossible to accurately compare North Carolina private school students who receive the vouchers with their public school counterparts who do not. It is also possible that the private school students who participated in the analysis were not representative of the average voucher student. That’s because the working paper only examined a small, non-random handful of voucher students (89 individuals, or 1.6 percent of all voucher recipients) who volunteered to be tested for the evaluation. In addition, just over half of the private schools attended by these 89 recipients were Catholic. Yet only 10 percent of all North Carolina voucher schools are Catholic.

The evaluation did use a statistical method called propensity-score matching to create a public school comparison group that was designed to be similar to the pool of private school volunteers. However, Nordstrom identifies five main flaws with this comparison:

The private school students who volunteered to participate in the evaluation were recruited by a pro-voucher advocacy organization, Parents for Educational Freedom in North Carolina. The evaluation does not clarify to what extent, if any, the organization cherry-picked the volunteers or their schools.

The public school students likely came from lower-income families than the voucher recipients. Evaluation authors said that they accounted for this difference by incorporating prior year’s test results into the analysis. But that assumes that income differences did not impact performance in the ensuing school year.

The public school students likely attended schools with higher poverty rates than the private school students would have been attending, absent the vouchers. Again, evaluation authors said that they accounted for this difference by incorporating prior year’s test results into the analysis, but that (again) assumes that the differences did not impact performance in the ensuing school year.

It is possible that the public and private school students had different levels of motivation when taking the test. While voucher recipients might have perceived that their performance could impact their ability to remain in their private schools, the public school students likely viewed the exam as a meaningless exercise.

The test used in the evaluation was not aligned to North Carolina’s Standard Course of Study. If it was aligned more closely with the private schools’ curricula, that could give the voucher recipients an advantage.

North Carolina’s voucher program is scheduled to grow by $10 million per year, to $144.8 million in 2027-28.
Yet as Nordstrom concludes:

North Carolina General Assembly lawmakers are about to conclude yet another legislative session without implementing meaningful evaluation and accountability measures on state voucher programs. Despite the N.C. State report, unfettered expansion of vouchers continues, and policymakers, educators, and parents still don’t know whether the program is working or not.

Categories Fraud, National Education Policy Center, North Carolina, Research, Vouchers

New Study: Attendance in Private Schools Has No Effect on Performance When Demographics are Controlled

By dianeravitch

July 13, 2018 //

A new study published by the peer-reviewed Educational Researcher by Professors Richard C. Pianta and Arya Ansari of the University of Virginia tests whether enrollment in private schools affects achievement when demography is controlled. The answer is no.

Here is the abstract:

By tracking longitudinally a sample of American children (n = 1,097), this study examined the extent to which enrollment in private schools between kindergarten and ninth grade was related to students’ academic, social, psychological, and attainment outcomes at age 15. Results from this investigation revealed that in unadjusted models, children with a history of enrollment in private schools performed better on nearly all outcomes assessed in adolescence. However, by simply controlling for the sociodemographic characteristics that selected children and families into these schools, all of the advantages of private school education were eliminated. There was also no evidence to suggest that low-income children or children enrolled in urban schools benefited more from private school enrollment.

Categories Education Reform, Poverty, Research, School Choice, Vouchers

National Education Policy Center: What Is the Evidence for the Success of No-Excuses Charter Schools?

By dianeravitch

April 19, 2018 //

Two researchers review a report recommending the widespread adoption of “no-excuses” methods and find the evidence inconclusive.

A. Chris Torres of Michigan State University and Joann W. Golann of Vanderbilt University review a report on “Charter Schools and the Achievement Gap.”

They write:

“A recent report, ‘Charter Schools and the Achievement Gap,” finds that, though charter schools on average perform no better than traditional public schools, urban “no-excuses” charter schools—which often use intensive discipline to enforce order—demonstrate promising re- sults. It recommends that these schools and their practices be widely replicated within and outside of the charter school sector. We find three major flaws with this conclusion. First, the report’s recommendations are based solely on the academic success of these schools and fail to address the controversy over their use of harsh disciplinary methods. No-excuses dis- ciplinary practices can contribute to high rates of exclusionary discipline (e.g., suspensions that push students out of school) and may not support a broad definition of student success. Second, the recommendation that schools replicate no-excuses practices begs the question of what exactly should be replicated. It does not confront the lack of research identifying which school practices are effective for improving student achievement. Third, the report does not address many of the underlying factors that would allow no-excuses schools and their practices to successfully replicate, such as additional resources, committed teachers, and students and families willing and able to abide by these schools’ stringent practices. Thus, while the report is nuanced in its review of charter school impacts, it lacks this same care in drawing its conclusions—greatly decreasing the usefulness of the report.”

How many parents are eager to subject their children to harsh discipline?

Categories Charter Schools, Education Industry, Education Reform, Privatization, Research

Economists with Common Sense about Education

By dianeravitch

March 24, 2018 //

Many of us on this blog criticize economists because it often seems that the only thing they value is scores on standardized tests. If they can’t measure a thing without precision, it doesn’t matter. They think they can measure teacher quality by student test scores, they can measure schools by test scores, they can measure students with test scores. As Daniel Koretz showed in his book The Testing Charade: Pretending to Improve Schools, the tests are misused and abused to make these judgments. They aren’t good enough to label students, teachers, or schools, and their misuse distorts the measures (Campbell’s Law).

Now a group of scholars seeks to rescue schools from the iron grip of standardized testing. (Among the authors is my favorite economist of education, Helen Ladd of Duke University.) They argue that test scores are not the only things that matter in education. They say that schools should be informed by evidence, not driven by it.

Decisions should be driven by what we value, what our goals are, not simply by test scores.

They write:

Although evidence clearly contributes to thoughtful policy-making, evidence cannot and should not drive policy decisions. When we make decisions, or policies, we are driven by a desire to achieve a set of goals. The role of data is to provide evidence on how our choices are likely to affect the realization of our goals. Evidence informs decisions so that, if the evidence is good and we interpret it well, the results of our decisions align better with what we value.

A challenge for many decision makers is to think clearly about the values they are seeking to realize. In education, decision makers are often motivated by the desire to improve student outcomes and increase educational equity. Yet both “student outcomes” and “equity” are vague terms. Which student outcomes, or combination of outcomes, are most valuable? Do we care about students’ understanding of trigonometry or their ability to run fast? Do we want to work towards all students having more equal cognitive skills or to increasing the skills of the least well off? Without more precise understandings of which outcomes we care about and which distributions of those outcomes are fair, decision makers lack orientation. Their decisions may end up relying on data about outcomes that happen to be available rather than about outcomes that align with their goals.

In a new book, Educational Goods: Values, Evidence, and Decision-Making (University of Chicago Press, 2018), we seek to spell out a set of educational values and distributive principles and to illustrate how they, along with a small number of non-education values, can be combined with the relevant evidence to improve education decision making. Two of us (Ladd and Loeb) are social scientists who bring a familiarity with the use of evidence, and two (Brighouse and Swift) are philosophers who operate in the realm of values.

This group of scholars is thinking differently. For example:

While educational goods and their distribution are central to education policy decisions, other values come into play as well. While it may at first seem like these additional values are too numerous and ill-defined to specify, in fact, only a small set of values—we identify five of them—typically come into play in education decision making. Think again of the possibility that equalizing educational goods would require extensive intervention in the family. Respect for parents’ interests limits the pursuit of distributive goals. Think of another independent value – what we term childhood goods. Childhood goods are the experiences that students have in childhood that contribute to their flourishing even if they do not build their capacities. These goods may include purposeless play, as well as the joys of learning or laughing. We may be unwilling to undertake an educational approach that develops students’ educational goods if it, in turn, makes the students miserable in the process. The other independent values—respect for the democratic process, freedom of residence and occupation, and other goods (e.g. heath care or housing)—may also put a brake on what should be done to pursue educational goods and their valuable distribution.

Categories Data, Economics, Research, Testing

Please Respond to This Research Survey about Opt Out

By dianeravitch

March 23, 2018 //

Reseachers at Teachers College, Columbia University, are conducting research about the Opt Out Movement.

Please consider participating in their survey if you are interested in the efforts of parents to keep their children from taking state tests as a protest against the overuse and misuse of standardized testing.

The survey was designed by two professors: Oren Piemonte-Levy and Nancy Green Saraisky.

For further information, you can contact:

Oren Pizmony-Levy, PhD
Assistant Professor of International and Comparative Education
Department of International and Transcultural Studies
Teachers College, Columbia University
525 West 120th Street
370 Grace Dodge Hall
Box 55
New York, NY 10027

Tel (office): 212-678-3180

Email: pizmony-levy@tc.columbia.edu
Website: http://orenpizmonylevy.com/

Categories Opt Out, Parents, Research

Jay P. Greene: The Disconnect Between Changing Test Scores and Changing Life Outcomes

By dianeravitch

March 15, 2018 //

Since the passage of No Child Left Behind, test scores have been defined by federal law as the goal of education. Schools and teachers that “produce”higher scores are good, schools and teachers that don’t are “bad,” and likely to suffer termination. The assumption is that higher test scores produce better life outcomes, and that is that.

In late 2016, Jay P. Greene produced a short and brilliant paper that challenged that assumption. I have fallen into the habit of asking myself whether the young people who are super-stars in many non-academic fields had high scores and guessing they did not. Fortunately, it is only in schools where students get branded with numbers like Jean Val Jean of “Les Miserables.” Outside school, they can dazzle the world as athletes, musicians, inventors, or mechanics, without a brand.

Greene writes:

“If increasing test scores is a good indicator of improving later life outcomes, we should see roughly the same direction and magnitude in changes of scores and later outcomes in most rigorously identified studies. We do not. I’m not saying we never see a connection between changing test scores and changing later life outcomes (e.g. Chetty, et al); I’m just saying that we do not regularly see that relationship. For an indicator to be reliable, it should yield accurate predictions nearly all, or at least most, of the time.

“To illustrate the un-reliability of test score changes, I’m going to focus on rigorously identified research on school choice programs where we have later life outcomes. We could find plenty of examples of disconnect from other policy interventions, such as pre-school programs, but I am focusing on school choice because I know this literature best. The fact that we can find a disconnect between test score changes and later life outcomes in any literature, let alone in several, should undermine our confidence in test scores as a reliable indicator.

“I should also emphasize that by looking at rigorous research I am rigging things in favor of test scores. If we explored the most common use of test scores — examining the level of proficiency — there are no credible researchers who believe that is a reliable indicator of school or program quality. Even measures of growth in test scores or VAM are not rigorously identified indicators of school or program quality as they do not reveal what the growth would have been in the absence of that school or program. So, I think almost every credible researcher would agree that the vast majority of ways in which test scores are used by policymakers, regulators, portfolio managers, foundation officials, and other policy elites cannot be reliable indicators of the ability of schools or programs to improve later life outcomes.”

I would add that Chetty et al did not establish a causal relationship between teacher VAM and later life outcomes, only a correlation. The claim that my fourth grade teacher “caused” me not to become pregnant a decade later strains credulity. At least mine.

Greene’s essay includes an excellent reading list of studies showing high test scores but no change in high school graduation rate or college attendance.

The Milwaukee and D.C. voucher studies that show a gain in high school graduation rate should note the high attrition rate from these programs, which inflates the graduation rate.

Imagine saying to a governor, I have a policy intervention that will raise test scores but will have little or no effect on life outcomes. Would they jump at the offer? Based on the political activity of the past 15 years, the answer is yes.

Overall, however, a seminal essay from a prominent pro-choice scholar.

Categories Charter Schools, Research, School Choice, Vouchers

Trump Seeks Massive Cuts to Clean Energy Research

By dianeravitch

January 31, 2018 //

In his budget proposal for 2019, Trump will ask for dramatic cuts to Research on behalf of clean energy.

He prefers fossil fuels. He likes nuclear plants too.

Nothing beats “clean coal.”

http://wapo.st/2DQ6FJU

Categories Environment, Research, Trump

Jersey Jazzman: What We Discovered About “Reform”in Newark: A Summary

By dianeravitch

December 17, 2017 //

Mark Weber, aka Jersey Jazzman, worked with Bruce Baker at Rutgers University to review the progress of the “reforms” (aka privatization and disruption) in Newark. This post is the first in a series that will summarize their findings.

The National Education Policy Center published a lengthy report written by Dr. Bruce Baker and myself that looks closely at school “reform” in Newark. I wrote a short piece about our report at NJ Spotlight that gives summarizes our findings. We’ve also got a deep dive into the data for our report at the NJ Education Policy website.

You might be wondering why anyone outside of New Jersey, let alone Newark, should care about what we found. Let me give you a little background before I try to answer that question…

In 2010, Mark Zuckerberg, the CEO and founder of Facebook, went on the The Oprah Winfrey Show and announced that he was giving $100 million in a challenge grant toward the improvement of Newark’s schools. Within the next couple of years, Newark had a new superintendent, Cami Anderson. Anderson attempted to implement a series of “reforms” that were supposed to improve student achievement within the city’s entire publicly-financed school system.

In the time following the Zuckerberg donation, Newark has often been cited by “reformers” as a proof point. It has a large and growing charter school sector, it implemented a teacher contract with merit pay, it has a universal enrollment system, it “renewed” public district schools by churning school leadership, it implemented Common Core early (allegedly), and so on.

So when research was released this fall that purported to show that students had made “educationally meaningful improvements” in student outcomes, “reformers” both in and out of New Jersey saw it as a vindication. Charter schools are not only good — they don’t harm public schools, because they “do more with less.” Disruption in urban schools is good, because the intractable bureaucracies in these districts needs to be shredded. Teachers unions are impeding student learning because we don’t reward the best teachers and get rid of the worst…

And so on. If Newark’s student outcomes have improved, it has to be because these and other received truths of the “reformers” must be true.

But what if the data — including the research recently cited by Newark’s “reformers” — doesn’t show Newark has improved? What if other factors account for charter school “successes”? What if the test score gains in the district, relative to other, similar districts, isn’t unique, or educationally meaningful? What if all the “reforms” supposedly implemented in Newark weren’t actually put into place? What if the chaos and strife that has dogged Newark’s schools during this “reform” period hasn’t been worth it?

What if Newark, NJ isn’t an example of “reform” leading to success, but is instead a cautionary tale?

These are the questions we set out to tackle. And in the next series of posts here, I am going to lay out, in great detail, exactly what we found, and explain what the Newark “reform” experiment is actually telling us about the future of American education.

Categories Corporate Reform, Education Industry, New Jersey, Newark, Privatization, Research