Well before the current crisis over value-added assessment, this ad hoc group warned that there were many reasons to doubt the value of test-based evaluation.
Since that report was published there have been many more demonstrations of the invalidity of student tests as a measure of teaching quality.
In time, it will be clear to everyone who cares about education that this is not a good way to judge teacher quality.
Diane Ravitch's blog 
I think anyone who is familiar with teaching and thought about how such a test could work would have understood this a long time ago. Still, it’s great to have good research to share.
Nevertheless, the desire of so many to boil everything down to competitions and tests may trump any rational study. Note the correleation between climate change deniers and free market capitalists:
http://www.telegraph.co.uk/earth/environment/climatechange/9503044/Climate-change-deniers-are-either-extreme-free-marketeers-or-conspiracy-theorists.html
LikeLike
Exactamundo❢
LikeLike
Agree completely with what you have said. And to reiterate: “it’s great to have good research to share.”
The “research” that absolutely proves the invalidities involved in the educational standards and standardized testing regime was written in 1997 by Noel Wilson-“Educational Standards and the Problem of Error.” Wilson, as a former test maker himself, has identified 13 sources of error in the standardized testing process that, any one of which, render the whole process completely invalid. I invite you to read my ongoing summarization and discussion of the study on my blog “Promoting Just Education for All” @ revivingwilson.org .
LikeLike
Is US ED is churning education data? http://www.sec.gov/answers/churning.htm
Data is a commodity.
LikeLike
Whenever the Chicago Teachers Union decides they have the best contract they can get at this time (and THEY are the ones who know best!!), as they have said, the fight for public education continues — everywhere. Their strike will have changed the conversation (yes!) and helped to level the playing field (bravo!) — but it remains for us to continue the struggle, together, everywhere. Here’s is some ammunition for the ongoing struggle over testing and “evaluating” teachers. It’s two years old, but this fight is a lot older than that, and this is good to use as we raise our heads and take on this battle, together.
LikeLike
The problem is we keep using “VAM” as a summative measure. I’ve been playing around with the idea of, after each major test, using prior achievement as a covariate (using simple regression and their prior year’s standardized test score). I coin those who exceeded the prediction as the “positive deviants” and have them share with us, the class, the secret to their success. I look at those who underperform in private and seek out patterns…if there are any. Ultimately….VAM works great as a formative tool…but is a abused when used summatively.
LikeLike
“Ultimately….VAM works great as a formative tool” VAM is pure USDA grade AA bovine excrement (BE). BE in BE out!
LikeLike
When Jonathan Knapp (SEA) & Mary Lindquist (WEA) brag about ‘collaboration’, AND, they let the housing of our families and the health care of our families be tied to 15 year olds filling in fancy scantrons – ARE they competent for the job of “leader”? Are they most interested in WHO is going to hire them AFTER they’re done “collaborating” ??
IF the NEA & AFT weren’t politically incompetent, (is that by design or by accident?)
IF the NEA & AFT weren’t politically incompetent there would be multiple layers of messaging against the lies of VAM – and 1 of the layers would be calling lying despicable scum what they are, instead of beating around the bush with dilettante speak. BTW, I said ONE layer of the messaging – unlike the politically incompetent dilettante cla$$ who define Diversity as everyone has different solstice celebrations and everyone spouts salon speak, I don’t insist that ALL messaging passes my purity tests. Unlike the dilettante cla$$, I don’t insist all conform to me or I’ll get the vapors and quit.
I do insist that there is more than dilettante speak, or, you dilettantes are on your own.
rmm
LikeLike
I never see references to the Tennessee Value-Added model… It’s been around for 20 years and is now 35% of a teacher’s evaluation. This comprehensive and cautionary report should be required reading for all policy makers and education professionals.
LikeLike
Tell me about it
Diane Ravitch
LikeLike
The major problem with Tennessee’s model in my mind is (a) it is (or at least was) “proprietary” and not subject to any scholarly scrutiny and (b) as Morgan (2002) pointed out 10 years ago “…a lack of training for teachers and administrators in how to understand the data reports ….
preventing schools and teachers from using value-added data to
improve student learning and achievement.”
Once again…VAM is a great way to guide practice in a formative sense. Powerful even. I’ve used it to help promote a growth-oriented mindset in my classroom by celebrating and learning from “positive deviants” (those who statistically have shown a lot of “value” added). The negative side can be useful as well of course. I have been using simple regression…not perfect but much better than analyzing averages or “change scores”.
VAM is a great guide … but a terrible master. In Navy parlance (I’m a vet) using VAM formatively throughout the year allows me to “correct my course” now-and-then instead of running aground later on.
Morgan, J. P. (2002). Multiple choices:Testing students in Tennessee.
Nashville, TN: Comptroller of the Treasury (a reference I use often).
LikeLike
“Once again…VAM is a great way to guide practice in a formative sense.”
Once again, VAM is 100% pure USDA grade AA bovine excrement (BE). BE in BE out!
LikeLike