John Thompson, historian and teacher, says that the Gates Foundation is fighting a losing battle to justify value-added assessment. At its root, he says, is an assault on public education, facilitated by a worship of data and a belief in the value of teacher churn.

He writes:
One of the Gates Foundation’s star value-added scholars, Dan Goldhaber, has voiced “concerns about the use of VAM estimates at the high school level for the evaluation of individual teachers.” Two years ago, he asked and answered “yes” to the question of whether reformers would have placed less emphasis the value-added evaluations of individual teachers if research had focused on high schools rather than elementary schools.
I once saw Goldhaber’s statement as “a hopeful sign that research by non-educators may become more reality-based.”
As the use of estimates of test score growth in evaluations becomes even more discredited, Goldhaber is not alone in making statements such as, “The early evidence on states and localities using value added as a portion of more comprehensive evaluation systems suggests that it may not be differentiating teachers to the degree that was envisioned (Anderson, 2013).”
So, what is now happening in the aftermath of the latest warning against value-added evaluations? This time, the American Educational Research Association AERA Council “cautions against VAM being used to have a high-stakes, dispositive weight in evaluations.”
The logic used by the nation’s largest education research professional association is very similar to what I thought Goldhaber meant when he warned against using various tests and models that produce so many different estimates of the effectiveness of high school teachers. The point seems obvious. If VAMs are imposed on all types of schools and teachers with all types of tests and students, then they must work properly in that wide range of situations. It’s not good enough to say we should fire inner city high school teachers because some researchers believe that VAMs can measure the quality of teaching with random samples of low-poverty elementary students.
Goldhaber now notes, “AERA’s statement adds to the cacophony of voices urging either restraint or outright prohibition of VAMs for evaluating educators or institutions. Doubtless, these stakeholders are genuinely concerned about potential unintended consequences of adopting these performance measures.”
However, Goldhaber and other supporters of corporate reform still twist themselves into pretzels in arguing that we should remain on their value-added path. Ignoring the effects of sorting as one of the factors that make VAMs invalid and unreliable for evaluating individuals, Goldhaber counters the AERA by illogically citing a couple of studies that use random samples to defend the claim that they can be causally linked to a teacher’s performance.

In other words, Goldhaber grasps at any straws to claim that it might not have been a mistake to mandate the risky value-added experiment before studying its likely negative effects. His bottom line is that VAMs might not be worse than many other inaccurate education metrics. And, yes, many things in education, as in all other sectors of society, don’t work. But, even if VAMs were reliable and valid for evaluating individuals, most people who understand school systems would reject the inclusion of test scores in evaluations because of the predictable and destructive policies it would encourage.

Moreover, Goldhaber is attacking a straw man. The AERA and corporate reform opponents aren’t urging a multi-billion dollar investment to scale up failed policies! My classroom’s windows and ceiling leaked, even as I taught effectively. But, that doesn’t mean we should punch holes in roofs across the nation so that all schools have huge puddles of water on the floor!
For reasons that escape me if the goal was improving schools as opposed to defeating unions, Goldhaber also testified in the infamous Vergara case, which would wipe out all California laws protecting teachers’ rights. He chronicled the negative sides of seniority, but not the benefits of that legally-negotiated provision. One would have thought that a court would have sought evidence on both sides of the issue, and Goldhaber only explored one side.
Goldhaber estimated the harm that could be done through “a strict adherence” to the seniority provision of “Last In, First Out” (“LIFO”). I’m sure it occasionally happens, but I’ve never witnessed such a process where the union refused to engage in a give and take in regard to lay-offs. More importantly, it once would have been easy to adopt the old union proposal that LIFO rights not be extended to teachers who have earned an “Unsatisfactory” evaluation. An agreement on that issue could have propelled a collaborative effort to make teacher evaluations more rigorous (especially if they included peer review.)
Reformers like Goldhaber ignore the reasons why we must periodically mend, but not end seniority. His work did not address the enormous social and civil rights benefits of seniority. It is the teacher’s First Amendment. Without it, the jobs of leaders who resist nonstop teach-to-the test will be endangered. Systems will have a green light to fire veteran teachers merely to get rid of their higher salaries and benefits. Without LIFO, corporate reformers will mandate even more mass closures of urban schools. Test scores will remain the ammunition in a war to the death against teachers unions. The poorest children of color will continue to be the prime collateral damage.
Even though he did not do so before testifying in Vergara, I hoped that Goldhaber would subsequently update his methodology in order to study both sides – both the costs and the benefits to students – of seniority protections. He has not done so, even though his new research tackles some other issues. In fact, I would have once been cautiously optimistic when reading Are There Hidden Costs Associated with Lay-offs? Goldhaber, Katherine Strunk, David Knight, and Nate Brown focus on the stress created by layoffs. They conclude, “teachers laid off and hired back to teach in the next school year have significantly lower value added in their return year than they had in years unthreatened by layoffs.” They find that the stress of receiving a lay-off notice undermines instructional quality and contributes to the teacher “churn” that especially hurts children in the poorest schools.
In a rational world, such a finding would argue for the reform of the education budgeting process that distresses educators – not for punitive measures against teachers who were blameless in this matter. In an even more rational world, Goldhaber et. al’s research would be used as an argument for more funding so that systems don’t have to cut it so close, and to provide support to teachers and students in stressful high-challenge schools.
By the way, I once faced such a layoff. It wouldn’t make my list as one of the thousands of the most stressful events of my career. The transparency of the process mitigated the uncertainty, minimized the chance of losing my job, and eliminated the chance that I would lose my career in an unfair manner. If Goldhaber and Strunk are really curious about the causes of teacher churn, they should visit the inner city and take a look at the real world that their metrics are supposed to represent. But, that is unlikely. Corporate reform worships at the idol of teacher churn. It is the cornerstone of the test, sort, reward, and punish policies that VAMs are a part of.
Goldhaber still seems to be sticking with the party line: Teacher churn is bad, except when it is good. We must punish teachers by undermining their legal rights in order to address the failings of the entire society. We must fight the stress fostered by generational poverty by imposing more stress on teachers and students in poor schools.
Once I believed that Gates-funded quantitative researchers were merely ignorant of the realities in schools. Maybe they simply did not know how to connect the dots and see how the policies they were advocating would interact with other anti-teacher, anti-union campaigns. Maybe I was naïve in believing that. But, at a time when the Broad Foundation is trying to replace half of Los Angeles’s schools with charters, we must remember the real danger of mandates for VAMs and against seniority in a competition-driven reform era where test scores are a matter of life and death for individual schools, as well as the careers of individual educators.
Every single rushed policy defended by Goldhaber may be a mere mistake. But, whether he understands it or not, the real danger comes from combining those policies in a top-down assault on public education.

Celia Oyler is a teacher educator at Teachers College, Columbia University.

In this post, she explains that the Chancellor of the Néw York Board of Regents, Merryl Tisch, does not understand how the teacher evaluation plan she snd 10 other Regents just approved works.

Tisch thinks she solved the problem of VAM mistakes by permitting teachers like Sheri Lederman to appeal ratings that are clearly wrong.

Oyler says that Sheri Lederman’s rating, egregiously wrong, was not an “aberration.” The whole system is flawed.

“What is extremely important for all New York State educators and families to understand is that the Chancellor of the Board of Regents does not understand a very basic aspect of a policy she has foisted upon us.”

One of the affidavits at the trial of the Lederman v. King case was filed by psychologist Brad Lindell.

His full affidavit is included in this post, which contains all the affidavits.

He sent the following note to me to explain his view of VAM in layman’s terms:

I am Dr. Brad Lindell, one of the affiants in the Sheri Lederman case who was present at the oral arguments on Wednesday. It was truly something to observe. You got the feeling that good was was going to come from the great work of Sheri and Bruce Lederman and from the experts’ opinions in so far as changing this broken VAM system. You got the sense that the judge was listening to the science about VAM and not just to the political rhetoric.

Just want to fill you in on something that was presented in my affidavit modified to give a clear and understandable example of the effects of poor reliability on a full-scale WISC intelligence test. If the same test-retest reliability from the teacher assigned yearly VAM scores (.40) was applied to the WISC full-scale to determine the 90% confidence interval, the range would be ridiculously large.

Examples

If a student scored a full-scale IQ of 100 (average) then the 90% confidence interval would be an 81 to 119. This indicates that there would be a wide range where the scores from repeated administrations of the WISC would be expected to fall for this student. One could not have confidence in the validity of a intelligence test with low reliability. Without adequate reliability, there can not be validity. This same holds true for VAM scores, whose reliabilities have been found to be notorious low.

The reliability of the WISC is generally in the .80 to .90 range. The 90% confidence intervals are generally in the +\- 6 range. So this same person with a 100 full-scale IQ would have a 90% confidence range of 94-106. Quite a smaller range.

This is why reliability is so important, which has repeatedly been shown to be low like .2 to .4 for year-to-year VAM scores. This is also why teachers year to year VAM score vary so considerably, like in the case of Sheri Lederman. Without reliability there cannot be adequate validity.