Daniel Katz of Seton Hall University explores the meaning of Sheri Lederman’s victory in court over New York State’s teacher evaluation system, the one promoted by former Commissioner John King (now Secretary of Education). He shows the complicated statistical calculations that produce “VAM” ratings and growth scores. Bruce Lederman, the attorney representing his wife in the proceedings, called them “a statistical black box.” It is not clear that anyone understands these models or can claim that they accurately measure teacher quality. This case is probably the first in the nation where a teacher has successfully overturned her rating.
Not only are these models difficult to impossible for teachers and most administrators to understand, they simply do not perform as advertised. Schochet and Chiang, in a 2010 report for Mathematica, found that in trying to classify teachers via growth models, error rates as high as 26% were possible when using three years of data, meaning one in four teachers could easily be misclassified in any given evaluation even if the evaluation used multiple years of data. Dr. Bruce Baker of Rutgers wanted to test the often floated talking point that some teachers are “irreplaceable” because they demonstrate a very high value added using student test scores. What he found, using New York City data, was an unstable mess where teachers were much more likely to ping around from the top 20% to below that and back up again over a five year stretch……
Equally important as the court’s recognition of arguments against value-added models in teacher evaluation, is the ground that was broken with the ruling. Ms. Lederman’s attorney (and husband), Bruce Lederman, sent out a message reported by New York City education activist Leonie Haimson which said, in part, ” …To my knowledge, this is the first time a judge has set aside an individual teacher’s VAM rating based upon a presentation like we made.” The significance of this cannot be overstated. For years now, teachers have been on the defensive and largely powerless, subjected to poorly thought out policies which, nevertheless, had force of policy and law on their side. Lederman v. King begins the process of flipping that script, giving New York teachers an effective argument to make on their behalf and challenging policy makers to find some means of defending their desire to use evaluation tools that are “capricious and arbitrary.” While this case will not overturn whatever system NYSED thinks up next, it should force Albany to think really long and hard about how many times they want to defend themselves in court from wave after wave of teachers challenging their test-based ratings.