If you are anywhere near Albany, New York, you should try to attend the oral arguments in the Lederman v. King case on August 12 at 10 a.m. in the court of Judge McDonough, Albany County Supreme Court, 16 Eagle Street, Albany, New York.
This is a major challenge to New York’s teacher evaluation system, which could have national implications.
Sheri Lederman is an extremely successful and respected fourth grade teacher in Great Neck, New York, who was rated “ineffective” even though more than twice as many of her students passed the state tests as in the rest of the state. Her lawyer is her husband Bruce Lederman.
Here is a description of the case.
Several national experts on teacher evaluation have submitted affidavits supporting Sheri Lederman, including Linda Darling-Hammond, Aaron Pallas, Sean Corcoran, and Audrey Amrein-Beardsley, asserting that New York State’s teacher evaluation system is incoherent.
Diane Ravitch's blog 
Maybe one of the Ledermans can weigh in on what the issue is for the August 12 hearing.
While I agree that VAM (and other test-based teacher evaluation methods) is a complete sham, I hope she has a stronger argument than just the fact that twice as many of her students passed the tests than the state average. Being from such an affluent area, her students would be expected to pass the tests at much higher rate than the state average. That doesn’t necessarily mean that she’s an “effective” teacher even if tests had anything to do with teacher “effectiveness”.
Dienne, your argument doesn’t make sense. Does this mean a teacher in a low socioeconomic school district or a teacher of students with special needs with students not performing well should be highly effective if they really are, regardless of their test scores or VAM? If so, then Lederman’s case is strong for the same reason. VAM and student performance has nothing to do with a teachers’ effectiveness which is exactly why this lawsuit is necessary!
Teachers from low socio-economic schools argue all the time (rightly) that their students’ low scores shouldn’t be held against them because poverty is a much bigger factor than teaching input. By the same token then, teachers in high socio-economic areas can’t take credit for their students’ high test scores. To the extent they measure anything, test scores measure affluence, so if Lederman’s whole argument is that she is an effective teacher because of her students’ high test scores, she’s going to lose in court because her affluent students are expected to get high test scores (at least, certainly higher than the state average). What I’m saying is that she’d better have something besides test scores to show that she is, in fact, an effective teacher.
Indeed. As I wrote last October:
“This district spends $33,000 per student (elementary school teachers can make as much as $140,000); 92% of that is funded by local property taxes. Only 6% of the district’s students qualify for a free lunch, 4% are LEP, 7% are Latino, and somehow, in a community in the heart of the NYC metropolitan area, a stone’s throw from the borough of Queens, just 1% are black. The median home value is $800,000 or so; the median household income is $90,000. Many of the residents are close-knit multi-generational Persian, Korean, and Chinese families who are passionate about education and who moved to Great Neck expressly for the schools.
“Perhaps the judge assigned to this case won’t have heard the argument that schools are only as good as their students, or that these tests are a proxy for family income, education level, and involvement, and he or she will be impressed by this 68-31% comparison. Otherwise, unless the teacher can prove she was somehow assigned a disproportionate share of the district’s scant number of at-risk students, 68% might not say what the plaintiff thinks it says.”
I do not get the general sense that Team Lederman will be making references to the works of Noel Wilson during the proceedings.
Hey, no digs against Wilson.
The law is the law, a hugely imperfect system of rules created and constantly amended by — people. And not always the all-around sharpest instruments in the little decked-out basement workshop. And then people have to abide by, analyze or argue against the letter and/or spirit of those laws, and then this is all often played out before people, juries, who know little about all of that. Then comes all of the drama, gamesmanship, grandstanding, etc. I don’t know how lawyers do it. Maybe FLERP can weigh in here. Right or wrong, ain’t much room for academic philosophy, though it may be the truth. If it doesn’t play well with the law or judges or juries, it’s a no go. But that says absolutely nothing about its validity or capacity to eventually become the law, as we evolve, as arguments are heard on high or within prominent skulls.
I’m under moderation, so generally not able to weigh in on anything with confidence it will get through.
As usual, Tim offers his dig against teachers and unions and a plug for the reform movement.
As usual, Tim never discusses anything about income redistribution and a reduction in Federal government corporate welfare so that money can be saved to pay for public schools. It’is always the teachers and civil servants who are sucking the life out of the rest of us.
It’s our fault, as we don’t watch and worship enough Fox News the way Tim does.
As usual, nothing from Tim about taxing the very wealthy and making them pay their fair share. Corporations included.
As usual, Tim’s critical thinking and statistics reporting are incomplete and are about as predictable as what one gets in a Big Mac and a Happy Meal.
For a really intellectual discussion on education and society, be sure to go through Tim’s McHappy drive through window of education banter. He might have a two-for-one offer for you, and how can you really resist?
I’m still trying to figure out how a professional basketball team with an ineffective coach can beat a high school team with an effective coach.
I’m still wondering how an Internet ideological troll that enjoys dropping stink bombs into conversations in the comments section can beat someone who knows what they are talking about and doesn’t blindly follow an ideology that has repeatedly failed.
And who said education was about ‘beating’ others? Poor analogy!
Chris in Florida,
Sorry that you interpreted my comment as that of a troll. It may be a poor analogy, but what I was trying to get at was in agreement with the observation that teachers in wealthy school districts are likely to appear more successful than their colleagues in impoverished districts given their students circumstances and backgrounds. Perform better may have been a more appropriate comparison.
Apology accepted and I offer my own in response.
The VAM scores are NOT based on the percentage of students passing. They are based on growth scores. Therefore, it doesn’t matter if she teaches in an affluent area or the number of students passing. The expectation of the VAM scores is for her students to show growth from the previous year’s assessment. This is why Ms. Lederman has a case. The state assessments compare test scores from year to year which is comparing apples to oranges. If a student scored a high 4 last year and a low 4 this year, that would negatively impact the teacher’s VAM score because it didn’t indicate growth. A classroom of high achievers is risky for a teacher because each year the scores must increase even when the difficulty level of the assessments is inconsistent.
I’m not sure that’s accurate. As I understand it, VAMs compare students’ actual performance not against their previous year’s performance, but against their *predicted* performance based on various characteristics and controlling for others. So the key is to beat the predicted performance, which is partly based on the prior year’s score, but also based on other variables. Unfortunately that’s the limit of my shaky understanding of this crap.
I think you’re right. In order to compare students’ scores year to year, they’d have to track individual kids as they move from school to school, place to place and I’m pretty sure that’s not being done (would love to see it done, though).
Dienne, you would not only have to track individual students (which computers can do) but you would have to believe that scores on standardized tests were the most important measure of education.
Right, I believe, but the high performers with lots of positive indicators behind them, as inhabit many of the more elite schools, would also have upwardly slanted predictions. So, it’s the same idea. It’s tough to improve at those altitudes, and one little snag would count a lot, negatively.
But if it’s tough to improve at those altitudes, the predicted score should reflect that toughness. Leaving aside whether how well it does that, I think that’s what the model tries to do.
FLERP, I think you should read the court filings about the case before you pass judgment. I did. Contact Bruce Ledermann at blederman@dagll.com
Not many people know more about teacher evaluation than Linda Darling-Hammond, Audrey Amrein-Beardsley, Sean Corcoran, Jesse Rothstein, Aaron Pallas, and Carol Burris, all of whom filed affidavits supporting Sheri Lederman.
I haven’t passed judgment on the case. That’s why I expressed hope that Bruce might be able to tell the readers here exactly what will be at issue at the August hearing. Do you know?
I also haven’t passed judgment on state VAM evaluations, in any of their forms, largely because I’m not competent to do so. I do know that they essentially measure actual scores against predicted scores, rather than simply against prior year scores. I also know that there are a lot of methodological concerns about the models that have been raised by VAM critics. Many of those concerns seem valid to me, but again, I’m not competent to really wade into the deep water.
I’m opposed to the use of VAM because they are extraordinarily arcane and obscure to the layperson. Employees should not be evaluated by a method that neither they nor their bosses understand. It will always breed dysfunction and distrust, regardless of how accurate or sound the method is.
I have read Bruce Lederman’s brief. I read it months ago. I will post it after I get a copy from him.
I’ve looked at a couple of those VAM formulas and I think that there is a very serious reason why the Wall Street moguls (Remember, value added was originally designed to predict — unsuccessfully, remembering the 2008 meltdown — the performances of companies on the New York Stock Exchange.) don’t want the public, especially not teachers, to understand it.
The algorithms predict the rate of change of one variable compared with another. Variables are represented by numbers, of course. The first variable in a function is last year’s test scores. One of the variables in each of the formulas is race. Another is family income. If your student is the white daughter of an attorney you had better raise those test scores a lot if you want to keep your job. If your student is the African-American son of a homeless family, his scores are not predicted to rise very much at all, so VAM winds up encouraging you to leave that child further behind.
That is why I am so serious when I say that my students are not numbers. They are not variables. Their progress should not depend on their race or their family income. Neither should my income depend on their race or their family income.
Using computer algorithms to predict human behavior is a dangerous path to walk for so many reasons. I do not think technology optimists and apologists will ever understand that.
Let me put it this way, FLERP:
At those altitudes there is little wiggle room. You would need highly precise instruments, that don’t exist and perhaps can never exist, to measure true, tiny gains or losses. And VAM is nothing if not totally imprecise and meaningless. If a top notch student were to totally tank, that could of course be measured, noted, noticed, etc., all sorts of ways. VAM would have about as much chance of recording it as a coin toss.
Further:
If you’re hovering near a high plateau or ceiling, it’s more likely or only possible for there will be slight downturns that do not signify anything real, only the fact there is a ceiling of some sort. Those false negatives may be picked up by crap mechanisms.
Akademos, that’s not really true. While it is true that against a hard ceiling in which significant numbers of students are answering every question correctly it is not possible for those students to improve performance, the PARCC/SBAC included more challenging questions to help differentiate. If virtually no student gets a “perfect” score, then every student can show growth.
The same applies to SGPs. Recall that at every level of the achievement test, SGPs produce an entire range of percentiles from 1-99. So to suggest that a top-scoring student cannot score an above-average percentile is simply false.
Maybe we should add these questions to the math portions of the PARCC/SBAC. Some of you might begin to argue once again that the questions are “too hard”.
Right. That’s why Tisch was floating the idea of letting the more elite schools out of the VAM jam.
I would really love to know whether as a student or a teacher what the “average” alternate universe version of myself would achieve as a student or teacher according to the Wizards at NYS.
VAM is more like a mathematical wish list for reality than it is science. We can never test its alternate universe propositions so any attempt to assess it is based on its own methods with no empirical proof.
Contrary to public misunderstanding [not accidental but deliberately fostered by rheephormsters], VAM doesn’t measure “something absolute” but “something relative to something else.”
Hence even by rigged rheephorm metrics teachers could be “highly effective” even if their students were only doing infinitesimally better than those of “highly ineffective” teachers. *Better? Worse? Think test scores.* Which recalls those [in]famous rheephormista critiques of NRT [norm-referenced tests] of yesteryear in favor of CRTs [criterion-referenced tests]. Phony tempests in phony teapots. The arguments du jour change as the targets of rheephorm change but there is still a toxic charge of vacuity in all of them…
I would add that the arguments of self-styled “education reform” lack legs but they’re missing a whole lot of other body parts too including (and especially) hearts and heads.
And yes, measure-to-punish is meant for OTHER PEOPLE’S CHILDREN. For THEIR OWN CHILDREN, only a bare minimum of what is necessary to remove obstacles on their way to ensuring a lifetime of advantage.
Thank you both for your comments.
😎
It is possible that the judge will rule on a point of law and not on VAM validity as other judges have. We may hear,”While it may not be fair, it’s the law as we did in other cases.” It also depends on the strategy of her lawyer. If that is the outcome, someone should look at other ways to attack VAM, either on validity or even as a civil suit for damages. I am not a lawyer, but I think creative routes may have to be explored, depending on the outcome.
Good question – what is the job of the state Supreme Court in this case?
At the federal level they usually intercede when lower courts get hung up on a question of interpretation – there is no preceding case that I am aware of.
I can only imagine it is in state Supreme Court because it is against a state law. SCOTUS can strike down a law based on pretty much constitutionality only or more narrowly interpret a law.
Does this court rule based on the constitution or state constitution?
Is their role mainly one of interpretation of the law or application as opposed to rights issues?
If the law was followed however ridiculous, if the court has no power to strike down laws, it may not be such a slam dunk.
I honestly do not know with a court at this level
A ruling by a state supreme court has implications for similar cases in all states.
New York’s State Supreme Court is different from other supreme courts:
https://en.m.wikipedia.org/wiki/New_York_Supreme_Court
“The Supreme Court of the State of New York is the trial-level court of general jurisdiction in the New York State Unified Court System. It is vested with unlimited civil and criminal jurisdiction, although outside New York City it acts primarily as a court of civil jurisdiction, with most criminal matters handled in County Court.[1] There is a branch of the New York Supreme Court in each of New York’s 62 counties.[1] Unlike in most other states, the Supreme Court is a trial court and is not the highest court in the state. The highest court of the State of New York is the Court of Appeals.”
Yes they have the right jurisidiction to hear the case, and she has merit to her being injured by the system.
However, the options aren’t just is VAM ok or not – in order to kill the law you’d need to prove that either the entire metric is erroneous or they could concede it was unfair and say we will tweak it more.
The real danger with VAM is the never ending promise that it can be made better without having to prove that improvements will have the intended effects which when applied evenly to infinite permutations of teachers and students, not a single one of which can be duplicated, is a mirage.
In any case, we will have to wait and see and hope the decision has far reaching positive outcomes.
I get what you’re saying, but it doesn’t help Ms. Lederman’s case. Presumably, she was found “ineffective” because her students’ scores didn’t show enough “growth”. It doesn’t help her to say, “well, yeah, but twice as many of them passed the tests than the state average”. VAM doesn’t require her to compete against the state average, it requires her to show “growth” with students who are already well above the state average.
Again, I’m not defending VAM – it’s complete junk science as has been demonstrated over and over again. But we have to be careful how we argue against VAM lest we accidentally give it more credibility in the public eye.
If anything her case points out the absurdity of the formula, but you never know what laws the judge will base his decision on and what his personal bias is.
Diane, is there any place where we can see the affidavits submitted by Dr. Amrein-Beardsley and Darling-Hammond? Thanks.
Virginiasgp,
I will post all the affidavits next week or over the weekend.