A new study of teacher evaluation finds that the more that value-added test scores count, the lower teachers are rated.
“Among the most important findings from this new study: When value-added scores are incorporated into evaluations, the ratings tend to go down. And the more weight a system puts on value-added scoring, the lower the scores are likely to be, the study showed.
“That’s because value-added scores tend to be relative measures, explained Kraft. Value added is generally “designed to just compare you to your peers,” he said. “Everybody can’t be good with value added.”
“On the other hand, with classroom observation scores, everyone can be excellent, he said.
“And that leads to a huge caveat in all of this: As it stands, despite the variation in systems, almost all teachers across the country continue to get positive ratings.
“That’s largely because observation scores make up the meat of most teacher evaluation systems, said Kraft. And as we know from previous research, principals tend to rate their teachers highly.”
Diane Ravitch's blog 
“The More Weight Given to VAM, the Lower Teacher Evaluation Ratings”
Isn’t that the point?
LikeLike
Such idiocy.
It’s one thing to have a sort of reality check for how students are faring, and to even place that, integrate that into an evaluation. But to have this system in which these numbers, hugely affected by school cultures, iniatiatives, etc., with no insulation from outside events or internal pranks, conspiracies, etc., form swords of Damocles . . .
Support people. Be real with people. You’ll get real results.
Threaten people, you get fake results. And everybody feels it, students, parents, etc. You get begrudging efforts, disgruntled employees and people daydreaming about how to expose and stop the madness, about retirement and embittered memoirs, about how much their pensions should be to buy their silence.
LikeLike
I WROTE THIS A DECADE AGOLearning not Teacher evaluation should be the emphasis of media
http://www.opednews.com/articles/Learning-not-Teacher-evalu-by-Susan-Lee-Schwartz-111001-956.html
LikeLiked by 1 person
saddest words these days: “I wrote this a decade ago…”
LikeLike
WOW, how CRAZY! So CALVANISTIC. People don’t do anything better in the long-term because of FEAR.
LikeLike
VAM or AGT — or whatever you call it — was never, NEVER, NEV-VER about “putting kids first,” as it has nothing, NOTHING, NUH-THING to do with “improving educational outcomes for children.”
From DAY ONE , the push for VAM was and is about union-busting, pure and simple.
Like a mortar round dropped into the middle of a platoon of soldiers, VAM indiscriminately murders those in the immediate radius, severely maims those just outside that radius, and causes the rest outside THAT radius to run and scatter away in all different directions .. effectively isolating all of them and destroying any collective voice for teachers.
This, of course, paves the way to privatizing schools, and profiting from that privatization, as — other than parent groups and other groups such as (recently) the NAACP — teachers’ collective voice (yeah, that means unions) is the only thing standing in the way of the disaster that school privatization brings:
— the de-professionalization of teaching;
— replacing live teachers with computers;
— cutting the funding for schools to enable privatization;
— charter schools that are unaccountable, non-transparent to the public, and which do not educate ALL the public, as they refuse to admit or later push out those students who are most expensive (Special Ed.) or most troublesome and/or less likely to do well on standardized tests (homeless kids, foster care, kids with disruptive behavior) and on and on.
In addition, VAM and all that accompanies VAM has devastated / devastates the curriculum, as the entirety of school days — or almost the entirety of school days — was/is spent on mind-numbing test prep in the only two subjects testes: language and math. Goodbye to all the essential elements of a full rich curriculum: music, dance, art, P.E., school performances, etc. I recently read about a school that was cancelling its holiday show because they needed the time spent on a holiday show to do test prep instead.
It’s a good time to again watch John Oliver’s takedown of the ridiculous over-emphasis and misuse of standardized testing:
Oliver points out how this pits students against teachers, with teachers saying, “Don’t f— me on this”, which the kids scores could determine whether or not those teachers are fired, or how much those teachers are paid.
You want to know how threatened the corporate ed. reform industry was by this piece of Oliver’s? Dozens of articles poured forth accusing John Oliver of “throwing students under the bus,” as Education Post’s $360,000/year Peter Cunningham does here:
http://educationpost.org/john-oliver-throws-poor-kids-under-the-bus/
Fred Klonsky had a great response to this:
Here’s the former Secretary of Ed. John King talking about how testing helps identify which schools need more funding or attention. Why, King says that standardized testing is “the key” to a “improving education and fixing inequalities”:
http://www.cleveland.com/metro/index.ssf/2017/10/all_those_tests_at_school_are.html
This is the same John King sends his own children to a private Montessori school which has NONE of this standardized testing. As with Campbell Brown, Arne Duncan, Michelle Rhee, Kevin Huffman, and countless other pro-testing creeps, King spends tens of thousands of dollars to keep his own children AS FAR AWAY FROM STANDARDIZED TESTING AS HIS CORPORATE-ED-REFORM-ORGINATED SALARY CAN PROVIDE.
I don’t have the link handy, but Peter Greene refuted this, asking (from memory, no an exact quote):
“Can anyone point me to one instance in which standardized testing led to more funding for schools, or lowering class size, or recruiting better quality teachers, or whatever?” Low test scores have only used as a pretext to close traditional public schools and re-open them as privately-managed charter schools.
Furthermore, as Greene and others has pointed out, studies definitively and unanimously show that classroom grades from those live teachers, and based on tests created by those live teachers …. are far more accurate in predicting a student’s success in college acceptances, completion of college, and career success.
In the movie CITIZEN KANE, Bernstein says, “There’s no trick to making a lot of money … if all you want to do … *is make a lot of money(.” It’s the same with high test scores. Test prep is the educational equivalent of steroids .. .you get short term positive results, it’s destructive in the long run.
As any testing expert will say, if you game the system with massive test prep, it invalidates any results you do get (looking at you, Eva Moskowitz!) Some say the standardized testing is invalid in all circumstances, though I don’t buy into that extreme view.
And then, of course, you just cheat by erasing and changing the answers on answer sheets — as was done in D.C., Atlanta, and countless other places.
LikeLike
I looked at the abstract of the study. It says:
“Using data from the Measures of Effective Teaching study, we conduct simulation-based analyses that illustrate the critical role that performance measure weights and ratings thresholds play in determining teachers’ summative evaluation ratings and the distribution of teacher proficiency rates.”
Sad to say this research is based on flawed data from the get-go, and a fiction —a statistical simulation. The MET studies were done by economists, most at Harvard, and funded by the Gates Foundation, about $64 million. The Researchers though they could just randomly assign teachers to classrooms according to a key assumption in calculating VAM. Not possible, as they soon found out from many breeches in their initial agreements with principals.
Then there is this. The classroom observations were not done in real classrooms. The raters looked at videos segments submitted by teachers who volunteered for the study in exchange for some video equipment.
The main observation schedule was from Charlotte Danielson, a protocol that has no reliability or validity for every grade and subject where it is used.
The VAM calculations in the MET study also differed from those in many states, including states where EVASS from SAS is used.
The promoters of this study also are also hell-bent on renewing attention to the 2009 New Teacher Project publication, the “Widget Effect: Our National Failure to Acknowledge and Act on Differences in Teacher Effectiveness.”
Recall that this document urged outcomes-only teacher evaluation and a system of stack ratings “normed regularly” around the outcome that is said to matter most, test scores ( p. 45).
For one of several informed criticisms of the MET studies, see http://nepc.colorado.edu/thinktank/review-MET-final-2013
Anyone who has taught knows that evaluations of teachers that make the most difference are those from a person whom you can trust as a professional, not someone committed to a dumb formula that even statisticians, especially statisticians, condemn as invalid and unreliable.
LikeLike
Human evaluations are far more authentic. As you stated, VAM was wrong from the start. Even if someone believes in the value of standardized tests, the results of those tests were never meant to be used to evaluate teacher performance. VAM is a secretive formula based on erroneous assumptions. The formula can be easily manipulated or tweaked to achieve the desired result in the same manner the cut scores on the standardized tests have been moved to meet the demand of a biased agenda. It is pure “reform” theater! What is tragic is that people have lost careers or pay due to junk pseudo-science.
LikeLike
Proponents of high stakes testing, such as Bill Gates, should be scientifically literate enough to realize if one is trying to measure one factor, such as teacher quality, all other factors must the same. One of the few situations would be to have two classes each made up of half a pair of identical twins, each with a different teacher.
LikeLike
And even your scenario with supposed “identical” twins does not hold water as the “gold” standard of supposed research protocol as there are still far too many variables that are unaccounted for.
LikeLike
Not to mention that there is not a person on this planet who can define a factor called “teacher quality” unless you accept student test scores as a valid indicator.
LikeLike
Gates used his personal capital to manipulate the destruction of the teaching profession. Gates weaponized his wealth to to trap teachers in a maze of his own design. Gates’ billions brought us the Common Core, VAM and Depersonalized Learning, his ultimate destructive goal.
LikeLike
It’s an extraordinary abandonment of science, humanity and common sense. The kind of thing one might have expected in the 50’s or 60’s. Now, it’s a matter of intent, willful ignorance and all. And a bit of anachronistic stupidity, I suspect, half naïveté, half ideology, religious or not.
LikeLike