We have been saying it for months, no, since 2009, when Race to the Top started.
Value-added assessment or value-added-modeling is not ready for prime time.
Now we have a technical paper by American Institutes for Research that says it:
VAM is not ready for prime time.
Here is the takeaway:
“We cannot at this time encourage anyone to
use VAM in a high stakes endeavor. If one
has to use VAM, then we suggest a two-step
process to initially use statistical models to
identify outliers (e.g., low-performing
teachers) and then to verify these results
with additional data. Using independent
information that can confirm or disconfirm
is helpful in many contexts. The value of
this use of evaluative change results could be
explored in further research efforts….”
Is anyone at the U.S. Department of Education listening?
Hello?
Diane Ravitch's blog 
No one has to listen. They are the Betas that serve the Alphas.
Considering that the use of any standardized test score is invalid in and of itself as proven by Noel Wilson*, and the VAM models all use student test scores to “evaluate” a teacher, considering that the use of any standardized test score for any purpose other than the one it is designed for is UNETHICAL**, the whole process is more than a sham. It is an INSANITY when one basis his/her “reality”, in this case the evaluation of a teacher, on errors, falsehoods and invalidities. Many people have been committed to mental institutions for being far less out of touch with “reality” than what the proponents of VAM suggest and do.
*See: “Educational Standards and the Problem of Error” found at:
http://epaa.asu.edu/ojs/article/view/577/700
“A Little Less than Valid: An Essay Review” found at:
http://www.edrev.info/essays/v10n5index.html
Click to access v10n5.pdf
**All major testing agencies all warn against using a standardized test score for anything other than what it was designed. A fifth grade math test is not designed to “evaluate” the teacher and to do so is UNETHICAL. I contend that even using any standardized test score is unethical due to the myriad errors involved in the whole process from design through the dissemination of the results as shown by Wilson referenced above.
This information has been known since 1997 when Wilson wrote his dissertation (and even then many have been sounding the warning against standardized testing from the beginnings of it-see Lehmann’s “The Big Test”). Most have turned a blind eye to Wilson’s most damning work. Best to ignore it, to confine it to the dustbin of history. But until we realize just how much we harm all children through the sorting and separating out processes that are standards and standardized testing and even classroom grading procedures we will continue to cause harm to the most vulnerable members of society, the children.
WAKE UP FOLKS!!
It’s wrong–bad theory and pedagogy. The reason is plain greed coupled with pure ego needs of the yahoos promoting VAM. And they are not mathematicians, too! Terrible, terrible!
Only if Wilson is actually right that all standardized testing, all numerical ranking even is invalid. But if he is right, then every teacher in the land would have to give up giving grades, and we know than none are going to do that. That’s their legal power hold over their students. Wilson’s idea invalidates any idea of ranking, any meritocracy. So, if school teachers won’t give it up for themselves, they HAVE to accept it in VAM assessments. Otherwise, we would be inconsistent, possibly hypocritical. He who lives by the grade, dies by the grade. Just another example of how public education DOES what it objects to when its own ox is gored. Or perhaps I misunderstand Wilson. What say you, my brown eyed colleague?
Harlan,
I have never understood why one would want to have that kind of “power” over students. I try to let them know that it is a game that is played by the system.
I try to let them know that grades are pure bullshit. And I agree with you that we do the wrong thing especially “when its ox is gored”.
But then again, if you understand what Wilson has to say you would agree with me that a lot needs to be done to “correct” what is going on in public education.
And I appreciate the brown eyed comment.
I completed a short research paper on VAM a month ago for a graduate class. There is no shortage of research going against it. Add this one to the pile.
The below link, while more about high-stakes testing, was one of my favorite articles. DiCarlo makes a great, but unfortunate, point:
“One can often hear people say that test-based accountability won’t “work.” The reality, however, is that it probably will. If we mold policy such that livelihoods depend on increasing scores, and we select and deselect people and institutions based on their ability to do so, then, over time, scores will most likely go up.”
http://shankerblog.org/?p=6835
I would bet that even if Dr. Sanders (VAM’s forefather) backed away from it’s use of education (his life work), reformers would still run with it.
In CT, our upcoming evaluations focus on student growth indexes. Of course, our first year of evaluations under the SEED process is our last year of CMTs. We’ll all look like bad teachers as we teach the Common Core and assess old curriculum standards.
I think VAM is next in CT. We need to get VAM into the conversation now, before it sneaks into legislation and reformers sell it to the public/legislators.
VAM us a FACTORY MODEL and NO PLACE for use in the classroom. Who are these ;)$&@ who make policy and have NOT A CLUE? Oh…forgot …. Follow the money. Always follow the money.
” If one has to use VAM…”
And therein lies the problem. That phrase is akin to saying “If one has to use astrology…” or “If one has to teach creationism…”
We are accepting what is admittedly junk science for purely ideological reasons. No one should have to use VAM – period.
“No one should have to use VAM – period.”
Exactly, and those that do use it should lose their professional license for gross negligence and incompetence.
Most of the researchers against it sadly end with that term…if one has to use VAM….
It seems rather odd that AIR, the DC “nonprofit” that has been making millions selling VAM algorithms in Race after Race to the Top state knowing that they would be used in high stakes decisions, would come out with a paper discrediting their use. Perhaps they are trying to protect themselves from future lawsuits?
AIR designed the NY VAM. And they admitted in their own work that it was not without problems. But full speed ahead.
Diane – I hope Governor Cuomo reads this paper and studies it before he gives his State of the State Address this week.
Marge
Don’t take bets on that.
Call me cynical, but VAM has never been intended to be anything other than a pseudo-scientific veneer for what is in reality a political question: are we to continue to provide universal public education, supplied by qualified teachers who are paid a living wage and benefits, have professional autonomy and due process rights?
The terminology itself reveals the biases and worldview of those who are imposing VAM. The phrase is a B-School term for, “The enhancement a company gives a product or service before offering it to customers.”
In other words, built into the language – sorry, I’m an English teacher and have this quaint notion that words matter – is the assumption that children are products and teachers are factors of production. And, in every business, it is imperative to try to lower the costs of production. Hence, VAM: a superficially objective measure (but with its political assumptions baked-in) that will provide the “facts” necessary for ensuring high turnover and reduced benefit obligations for school systems.
STOP. STANDARDIZED. TESTS. in. 2013. Parents, opt your kids out!
If you want sense do not talk to them as their ideological financial agenda for their puppet masters is all they care about. Just look at the true history of Obama and Duncan in education. What more do I need to say unless, as it appears, most do not know the truth only what is told to them even though it is right there in their face if they just spend a few minutes and look.
Reblogged this on Transparent Christina.
“If one has to use VAM, then we suggest a two-step process to initially use statistical models to identify outliers (e.g., low-performing teachers) and then to verify these results
with additional data.”
I believe someone on this blog has pointed out the pressure being placed on evaluators to have their observational data match the VAM data.
Teach,
So very true!
I have looked at all of the models on the NJ DOE website. One of them openly penalizes principals when the achievement (an odd mix of NORMATIVE test scores) does not match the performance (observation, walkthroughs… Ect..)! It is called, (I laughed out loud when I dug this up) INCONGRUENCE!
The whole thing is @&$ing incongruent.
But, our Commissioner of Education is a BROADIE with ethical challenges so no truth will slow down the destruction here in NJ.
Good luck where you are.
Is anyone at the U.S. Department of Education listening?
Hello?
No, neither listening, nor reading. So much for educated decision makers.
Good!!!