Recently, gubernatorial candidate Cynthia Nixon issued a press release calling for the repeal of the state teacher evaluation system, which links teacher evaluation to state test scores of their students.
Almost immediately, the State Assembly (in Democratic control) announced that it was writing a bill to revise test-based teacher evaluation. The Assembly bill passed overwhelmingly, but it was a sham. Instead of repealing test-based teacher evaluation, it said that districts could use the test of their own choosing to evaluate teachers, so long as the test was approved by the State Commissioner. That does not repeal test-based evaluation, and critics warned that there might be “double-testing,” once for the state tests, another time for local tests.
Now that the bill has moved to the State Senate, the Republican leader John Flanagan has said he will slow down movement on the bill because no one wants “double testing.”
In the one instance where the state’s teacher evaluation system was brought to a court, by lawyer Bruce Lederman on behalf of his superstar wife, Sheri, a fourth grade teacher on Long Island, the judge said that the system was “arbitrary” and “capricious” and threw out her rating.
When the system was first adopted, Governor Cuomo wanted 50% of a teacher’s rating to be based on student scores. In 2013-2014, when the first results of the rating system were reported, 97% of the state’s teachers were rated either “effective” or “highly effective.” In New York City, in the first year, 93% were rated in the two highest categories. By 2016-2017, 97% of New York City’s teachers were also rated either “effective” or “highly effective.”
Chalkbeat reviewed the controversy over the state teacher evaluation system and wrote:
The battle lines were redrawn again in 2015, when state lawmakers — led by Gov. Andrew Cuomo — sought to make it tougher for teachers to earn high ratings. The new system allowed for as much as half of a teacher’s rating to be based on test scores.
But that plan was never fully implemented. Following a wave of protests in which one in five New York families boycotted the state tests, officials backed away from several controversial education policies.
In late 2015, the state’s Board of Regents approved a four-year freeze on the most contentious aspect of the teacher evaluation law: the use of students’ scores on the grades 3-8 math and English tests. They later allowed districts to avoid having independent observers rate teachers — another unpopular provision in the original law.
In short, the Opt Out movement caused the state to call a moratorium even as Governor Cuomo and the legislature were demanding tougher evaluations.
Given the fact that the test-based evaluation system has not worked (97% of teachers are doing just fine, thank you), given the fact that a full-blown court challenge presented as a class action is likely to get the whole system declared invalid, and given the fact that there is a growing teacher shortage, given the fact that the American Statistical Association declared “value-added” evaluation” inappropriate for individual teachers, why not repeal test-based evaluation altogether?
Let school districts decide how to evaluate the teachers they hire. Let them decide whether to adopt peer review, principal observations, or some combinations thereof.
The current system is useless and pointless. It does not evaluate teachers fairly. It is expensive. It attaches high stakes to tests for teachers. It has no research to support it.
When in doubt, throw it out!
The quick legislative nonaction was all for show, to make it look like Cuomo cares.
Now they will do nothing until the election but Cuomo can say he started the ball rolling, even though it is just rolling around in a circle in a bowl.
Rolling around like a roulette ball, eh!?!
Our middle school students just finished 2 Days of state ELA and 2 days of state math- scores for which don’t count for anything. Now our students are taking 3 day long local assessments which won’t count for students but will. For teachers. On top of that, English Language Learners will be tested for 4 additional days. Then, we begin finals. Fun days!
aie yie yie. such a waste. what breathtaking opportunity costs!
Evaluation Session Stream of Consciousness Rant
OK, you are sitting in your year-end evaluation session, the freaking EIGHTH such session you’ve had this year, as though you had nothing better to do, and you’ve heard from every other teacher in your school that his or her scores were a full level lower this year than last, and so you know that the district office has leaned on the principal to give fewer exemplary ratings even though your school actually doesn’t have a problem with its test scores and people are doing what they did last year but a bit better, of course, because one grows each year as a teacher–one refines what one did before, and one never stops learning.
But you know that this ritual doesn’t have anything, really, to do with improvement. It has to do with everyone, all along the line, covering his or her tushy and playing the game and doing exactly what he or she is told, going all the way back to Bill Gates, who, being a God, evaluates but is not evaluated, who did stack ranking at Microsoft because, admit it, he was probably on the spectrum and didn’t know better and, since behaving in this appalling way made him incredibly wealthy, it must be right, huh? And, at any rate, everyone except the politicians and those paid to think otherwise knows that the tests in ELA are not actually valid or reliable and that’s not really the issue at your school because, the scores are pretty good because this is a suburban school with affluent parents, and the kids always, year after year, do quite well.
So whether the kids are learning isn’t really the issue. The issue is that by means of the latest magic formula pushed by the district and some InstaEduPundit, each cohort of kids is supposed to perform better than the last–significantly better–on the tests, though they come into your classes in exactly the same shape they’ve always come into them in because, you know, they are kids and they are just learning and teaching ISN’T magic. It’s a lot of hard work. It’s magical, sometimes, but its’ not magic.
So, the stuff you’ve been told to do in your “trainings” (“Bark. Roll over. Sit. Good Boy”) is pretty transparently teaching-to-the-test because some people, astonishingly, continue to believe, after years of evidence to the contrary, that that’s a way that one might actually meet the insane demand that each cohort will be magically superior to the last, but you feel in your heart of hearts that caving to this idiocy, this crowd madness, would be JUST WRONG, that it would short-change your students to start teaching InstaWriting-for-the Test, Grade 5, instead of, say, teaching writing. And despite all the demeaning crap you are subjected to, you still give a damn.
And you sit there and you actually feel sorry for this principal because she, too, is squirming like a fly in treacle in the muck that is Education Deform, and she knows she has fantastic teachers who knock it out of the park year after year, but her life has become a living hell of accountability reports and data chats to the point that she doesn’t have time for anything else anymore (she has said this many times), and now she has to sit there and tell her amazing veteran teachers who have worked so hard all these years and who care so much and give so much and know so freaking much that they are just satisfactory, and she feels like hell doing this and is wondering when she can retire.
And the fact that you BOTH know this hangs there in the room–the big, ugly, unspoken thing. And the politicians and the plutocrats and the mendacious twits at the Thomas B. Fordham Institute and that smarmy know-nothing David Coleman and the Secretary of the Department for the Privatization of Education, formerly the USDE, and the Vichy education guru collaborators with these people barrel ahead, like so many drunks in a car plowing through a crowd of pedestrians.
Brilliant, sad, and true.
Thank you, Laura, and thank you for all the brilliant research that you post here.
I lived it. I left it. Could not take the madness any longer.
Stand up and take a bow, Bob Shepherd. Excellent commentary.
Thank you, Zorba.
“drunks in a car plowing through a crowd of pedestrians.”
Yes, that pretty well sums it up, unfortunately.
The only thing more dangerous than a self-driving Uber car is a Deformer-driving Uber car.
“drunks in a car plowing through a crowd of pedestrians.”
Yes, and calling it “the civil rights movement of our time.”
Ah, Bob Shepherd, so good to hear from you regularly again. I was missing your gems such as this succinct description of teacher-training: “Bark. Roll over. Sit. Good Boy.”
Agreed. I’ve often thought, if “state ed” said to teachers, go out in front of the school, get down on all fours and bark like dogs, how many people would actually do it?
Well said, Bob. That’s for getting this all into words.
Bob: you have nailed the experience of anyone in the present system. Somebody said the job of middle management is to cushion the blow of bad decisions one level above you. Good principals must cynically ignore their orders. What a system.
“[T]he job of middle management is to cushion the blow of bad decisions one level above you.” Oh my, that is well said!
Bob,
If I fully credit your name, would you be willing to allow me to read your missive if I am speaking in public? Please let me know either way. I thought your tone and tenor stylistically – as well as the substance – were amazing. It has some theater in it, but it has the truth at its core.
LOVED it! Absolutely LOVED it!
Robert, that’s so very kind of you. Please do share this. Thanks you!!!
The test-based “Value-Added Method” (VAM) of evaluating teachers and now principals, too, has been “slammed” — quoting The Washington Post — by the very people who know the most about data measurement: The American Statistical Association (ASA). The ASA’s authoritative, detailed, VAM-slam analysis, titled “Statement on Using Value-Added Models for Educational Assessment” and has become the basis for teachers across the nation successfully challenging VAM-based evaluations.
Even though it’s anti-public school and anti-union, the Washington Post said the following about the ASA Statement: “You can be certain that members of the American Statistical Association, the largest organization in the United States representing statisticians and related professionals, know a thing or two about data and measurement. The ASA just slammed the high-stakes ‘value-added method’ (VAM) of evaluating teachers that has been increasingly embraced in states as part of school-reform efforts. VAM purports to be able to take student standardized test scores and measure the ‘value’ a teacher adds to student learning through complicated formulas that can supposedly factor out all of the other influences and emerge with a valid assessment of how effective a particular teacher has been. THESE FORMULAS CAN’T ACTUALLY DO THIS (emphasis added) with sufficient reliability and validity, but school reformers have pushed this approach and now most states use VAM as part of teacher evaluations.”
The ASA Statement points out the following and many other failings of testing-based VAM:
“System-level conditions” include everything from overcrowded and underfunded classrooms to district-and site-level management of the schools and to student poverty.
A copy of the VAM-slamming ASA Statement should be posted on the union bulletin board at every school site throughout our nation and should be explained to every teacher by their union at individual site faculty meetings so that teachers are aware of what it says about how invalid it is to use standardized test results to evaluate teachers or principals — and teachers’ and principals’ unions should fight all evaluations based on student test scores with the ASA statement as a good foundation for their fight.
A copy of the VAM-slamming ASA Statement should be posted on the union bulletin board at every school site throughout our nation
yes yes yes
Audrey Amrein-Beardsley et al wrote a paper that expounded upon the ASA report.
https://www.tandfonline.com/doi/full/10.1080/2330443X.2016.1164641
I think one thing that one has to bear in mind when reading such reports is that these are scholarly reports, which means the authors are reserved in their statements.
One can not expect the authors to come right out and say that “VAM should not be used to evaluate individual teachers,” even though it is quite clear from the points they make (eg, about low reliability of VAM for ranking individual teachers) that that is effectively what they are saying.
To someone who lacks an understanding of the relevant statistical issues (eg, standard error, correlation, variance, etc), the ASA report might seem to go fairly easy on VAM.
But to anyone who actually understands those issues, it is clear that it is actually quite a scathing critique of the use of VAM for evaluating individual teachers, particularly for high stakes purposes.
This is per Dr. Audrey Amrein-Beardsley: “In the simplest of terms, all [growth] models can be called student growth models (SGMs), with VAMs being the most popular and sophisticated of the bunch and the least sophisticated albeit still popular being the student growth percentile (SGP) model. Some call the SGP a VAM, but the aforementioned folks are usually quick to differentiate. Others call all of these VAMs, but VAMmers are also oft-quick to note that, indeed, there are differences. Using the SGM as the mother term is the best bet unless, indeed, any differentiation is warranted….
Notwithstanding, really any SGM (including any VAM, SGP, or other model/approach) can be used with really any set of tests…, not without a serious set of issues, however (i.e., there is a long list of issues, especially when “alternative assessments” that are much less reliable and valid are in play). These issues matter less, though, if states/districts are not encouraging or requiring the attachment of high-stakes consequences to any SGM output. Put differently, if the output matter for things like teacher tenure, merit pay, probation, termination, etc. these issues become much more serious, and much less defensible in court.”
FYI, after the Sheri Lederrman case, NYSED moved away from the VAM model and adopted the Student Growth Percentile model. See Explaining Student Growth Scores to Teachers and Principals 2016-17 Frequently Asked Questions August 2017; Prepared for the New York State Education Department by Education Analytics, Inc. under contract C-012798; at page 5: https://www.nctq.org/dmsView/2016-17-growth-scores-faq
This shift continues to couple student test scores to teacher evaluations for high-stakes purposes. Nuf said.
https://www.tandfonline.com/doi/full/10.1080/10627197.2018.1449634
Comments are making good points as did Professor Ravitch. Please sign our petition to repeal the teacher and principal evaluation laws.
https://petitions.moveon.org/sign/repeal-nys-teacher-evaluatio?source=embedhomepage
SGPs are are actually worse than VAM (if that is indeed possible), not better because SGP was never designed to evaluate teachers.
Bruce Baker of Rutgers wrote about that here
https://schoolfinance101.wordpress.com/2011/09/02/take-your-sgp-and-vamit-damn-it/
But I suspect that most of those pushing this stuff either dont know or dont care about what people like Baker say.
Most of them (people like Arne Duncan) are undoubtedly simpky too dumb to understand it. And the few Deformers who are smart enough (eg, Bill Gates) would never admit that they had read Baker’s stuff because then they would have to admit that their policies are junk.
@SomeDAM Poet: Thanks for posting this link; it’s highly informative. And you are correct — most people don’t know or don’t care that student growth models are junk science, and that, accordingly, student test scores should not be used to calculate student growth for purposes of high-stakes teacher evals.
Your link is an interesting article. The shortcomings of SGPs need to be widely disseminated. Otherwise, we are doing a disservice to teachers and students. In reading this, I was reminded of the testing and measurement course I took to be certified as a reading teacher in NYS. I remember we were told to never assume the percentile score on one measurement is valid on another measurement. We were told we always had to go to the scale scores for the individual test in order to determine percentile. Politicians need to start talking to experts in educational assessment before they over generalize a falsehood and disseminate the misinformation. Once again SGPs were never intended to be used to evaluate teachers.
What is very telling is that the article by Baker is from 2011 — 7 years ago.
And look at the explosion in the use of VAM and SGP since then, with pretty much a total disregard by policy makers for the input of actual experts like Baker and Amrein-Bearsley.
And the same pattern was repeated with standardized testing and Common Core and charters. No evidence and no expert opinion required ahead of time. In fact, any contradictory evidence was ignored or shoved asside.
This indicates a problem that extends far beyond the realm of education and actually threatens the very existence of our country: the disregard for science and facts.
As Nobel prize winning physicist Richard Feynman once said “For a successful technology, reality must take precedence over public relations, for Nature cannot be fooled”
The same thing applies to a policy like VAM. You can only avoid the facts for so long before it comes back to bite you. We are already seeing the “bite” in the form of teacher disillusionment and teacher defection.
I know of at least one math teacher who used to comment here who left because of VAM, testing and the other evidenceless, ideologically based policies that he was being forced to swallow.
I am sure there are probably many many more because even the most dedicated teachers can only take so much abuse before they walk.
Politicians and their overlords are grasping at straws trying to figure out a way to pin scores on teachers. They are so eager to do so that they are make fake assumptions that will ultimately end in more lawsuits.
I am sick of the ego needs, the stupidity, the arrogance, and the pure toxicity of “those in charge” and who “play for pay. “
Thank you, big bucks and politicians. May you all get your just dues.
Amen
It should all be thrown out, true. But if a local bargains not to use scores from tests in any manner then no test scores will be used in a teacher’s APPR. This is a good thing. Where am I wrong?
The revised law does not allow districts to use no tests. The districts must pick a test from the Commissioner’s list of approved tests. Not sure what is left to bargain about. What % the tests count?
The % that the test counts is largely irrelevant as Cathy O’Neil points out
https://mathbabe.org/2016/09/30/the-one-of-many-fallacy/
Audrey Amrein-Beardsley notes the same thing in the paper I linked to above.
Cathy has written many blog posts on VAM and let’s just say that as a professional, Harvard PhD mathematician, she is not impressed with VAM
https://mathbabe.org/?s=VAM
Should really be what % of the total teacher evaluation score the VaM score counts for
The % is largely irrelevant in that VAM “drives the bus” no matter the % of weight given to VAM.
That is because most of the variation in evaluation scores is in the VAM score, so the differentiation between effective and ineffective teachers is mainly dependent on the VAM score part of the evaluation and not the other parts even though those other parts may make up a much larger piece of the pie.
There is a looming teacher shortage in New York state. Districts in the capital region are scrambling to find substitute teachers. There hasn’t been a student teacher in my school in 3 years. Several colleges are reporting that the number of students getting into the Education field is drastically down. The baby boomer generation is retiring in droves in the next few years. Just the tip of the iceberg. We have Andy and his buddies to thank for this mess. He has made our profession frightening and unappealing to say the least.
Ok, so this is how dumb it is. There are New York colleges now offering a TESOL certificate so that teachers who would like to become certified in ESL (or ENL as it is now called) can do so with about 15 credits. Imagine how many physics or social studies teachers out there are saying to themselves, “Gee, I’d like to get certified in ENL!” Maybe….one person if that???? BUT there are foreign language teachers (like me) who might like to teach ELLs. Most of us are teaching a foreign language that is NOT our native language. Let us be automatically certified to teach ENL or have a short, doable course. Newer foreign language teachers teachers coming in (I imagine the numbers are dwindling.) usually have ENL certification in addition to their foreign language certification because the coursework is very similar. Older teachers like me who have a Master’s Degree in the foreign language they are teaching are few and far between. So, give us old-timers a break and let us teach ENL without going back to school for a year or more! It’s a logical solution to the shortage of ENL teachers.
This is a disgrace. When I took my masters degree in ESL, the content was more or less applied linguistics. This solid background served me well. Frankly, even some of the newer masters degree were a lot less rigorous than the preparation I had. I worked with many student teachers over the years. I kept trying to get the colleges to add some reading courses to their programs. I went back to get certified as a reading teacher because I had so many students that could not read in L 1. Sadly, the false belief is that if you speak English, you can teach it.
I tend to think this will eventually have to be decided in the courts because with few exceptions, our politicians don’t consult with real educational experts.
Several judges (eg. In the Lederman and Houston case) have already shown far more interest than politicians in what actual experts on VAM (like Audrey Amrein-Beardsley) have to say rather than what posers like Thomas Kane have to say. This is a good sign, but means that teachers will probably have to continue to seek redress in the courts.
Considering that using the results of a test for evaluating anything other than what the test was designed to assess is UNETHICAL. . . .
. . . Is it that hard to figure out what is wrong with using student math or ELA test scores or any standardized test score to evaluate a teacher?
It’s just such a waste. How much time and money have state legislatures spent on teacher measuring schemes over the last ten years and WHY did they all follow this path like lemmings?
My God, why bother electing these people if they never do their own thinking?
Who decreed that this was Job One for every state legislature in the country? What AREN’T they doing when they adopt these ed reform priorities to the exclusion of all else?
I think we are on the 5th different scheme in Ohio. Even if I wanted to use the Ohio teacher measuring scheme to determine something or other I couldn’t- I have no idea what it is, today.
Hire new people. Clean house. These people are lost and they can’t find their way back.
They followed this path because the 1% wants to destroy public schools. Causing a crisis among teachers is a sure way to get the teacher shortage we now face, and this shortage will undermine public education.
How did the entire country fall victim to the theories of three economists and can we possibly break free or are we doomed to follow them forever?
Are there other economists? Maybe we should find a few more and ask them.
All this baloney about “reinventing education” and it’s the same people and theories over and over and over, for decades. There’s no “debate” in ed reform. They quibble over tactics and methods but there’s no real debate over the basic tenets of the religion.
They magnify tiny differences- “charters should be non-profit!” “no, charters should be non-profit AND for-profit!” and this somehow passes as a high level intellectual debate when it’s management details and housekeeping.
Among 50 states one would think there would be ONE who would break free and go in a completely different direction, if only for the sake of an experiment.
The Reformation”
Reform is a religion
With Friedman as its God
And Billy Gates as Profit
To follow and to laud
With Charter as the chapel
Where people kneel and prey
With hymnal pads from Apple
To rapture them away
The Fundamental tenet
The key to Heavenly Gates
Is righteousness of market
That’s sealing all our fates
Very witty use of the word “prey.” Nice.
Another gem, SomeDAM!
What they are doing looks to be pretty clear. They aren’t getting rid of the instruments of public education destruction. They’re simply deflating the program temporarily until some future point where they develop the political capital to start up the crusades anew.