In a startling development, the State Board of Education in Tennessee made clear at its meeting on Friday that it may eliminate or modify value-added measurement (VAM).
State Commissioner Keven Huffman was stunned.
Tennessee is the state where VAM got started, launched in the 1980s by agricultural statistician William Sanders. Based on his experience, Sanders assumed that it was possible to hold all other variables constant and attribute the rise or fall of test scores to teachers. Most social scientists understand that children are not corn, and it is impossible to hold all other variables constant. But Sanders now has a consulting business, and his methods are proprietary information, c.osely held.
In the future, if the board sticks to its guns, VAM will not play a part in deciding whether teachers may be licensed to work in the state of Tennessee. Audrey Amrein-Beardsley, an expert on VAM, celebrates that major victory here. Beardsley includes links to the three YouTube videos that the TEA shared with the board, called “The Trouble with TVAAAS.”
Given this background, the discussion at Fridays board meeting was indeed welcome to critics of test-based accountability, which has failed wherever it has been put into place.
Joey Garrison of the Tennessean reports:
” At its meeting in Nashville on Friday, the board stepped away from the new policy, promising an April rewrite eliminating learning gains as the overriding factor in whether teachers can work in Tennessee. The state’s educators claimed victory after a three-year pounding that also ended the promises of contract negotiations and annual raises, then tied their tenure to student test scores.
“Education Commissioner Kevin Huffman, who backed the original policy, and state education board Chairman Fielding Rolston downplayed the vote, characterizing it as a small and appropriate tweak before the policy takes effect in 2015. Education reform advocates took the same tack, pointing out that basing licenses on overall teacher evaluation scores, which include learning gains but give more weight to principal observations, is still progress….
“But the vote coincides with a bipartisan bill gaining ground in the legislature this session. The Educator Respect and Accountability Act, sponsored by Rep. Matthew Hill, R-Jonesborough, and Sen. Mike Bell, R-Riceville, would prevent the state from yanking teachers’ licenses based on “any statistical estimate utilizing standardized test scores.”
Huffman, who is heartily disliked by both teachers and administrators (nearly half the state’s district superintendents signed a petition decrying his top-down, non-collaborative style), was clearly outraged by the board’s second thoughts but leveled his criticism at the union. (Tennessee withdrew collective bargaining rights in 2011, and the TEA is weak.)
He said “he was flummoxed to see the union withdrawing its support of any use of value-added scores in evaluations and licensing after the state received a $500 million federal Race to the Top grant in part based on it.
“This is a very significant change in their position, and to be frank, I find it incredibly hypocritical that TEA would support the inclusion of value-added scores when there’s a bunch of federal money at the table and then turn around a few years later when that money is no longer going to be around,” he said.
“In a media release, the 46,000-member TEA countered that it endorsed the Race to the Top plan under former Gov. Phil Bredesen’s administration, but Huffman and Gov. Bill Haslam didn’t honor the agreements.
“Since Bredesen’s administration, teachers have complained of abysmal morale over the loss of union contract negotiations and use of student test data in their evaluations — even if the subjects they teach aren’t tested.”
Another way to look at the change of view of the board and the teachers is that they know now that VAM doesn’t wrk, that it is inaccurate, and they are acknowledging the realities.
CHILDREN ARE NOT CORN….Finally!!!!!!
“Most social scientists understand that children are not corn, and it is impossible to hold all other variables constant.”
Sheesh, even in corn, you can’t hold variables equal!
lol..true
This could be a ploy to quiet the crowd..and then it is “business as usual”….
I do not trust these people anymore..
Neanderthal is correct to be skeptical of the final outcome. Huffman and his powerful allies will not allow VAM to stop being used for teacher licensing. It is very ironic that TEA went along with RTTT and now has their nose rubbed in it by Huffman. Once again it shows TEA’s commitment to money and member dues over anything else. If the state board becomes the savior of public schools in TN, that qualifies as a miracle. Sorry to sound so negative, but Tennessee is at risk of destroying its public schools, and it’s so sad.
I agree. Can’t trust them one iota.
“was clearly outraged by the board’s second thoughts but leveled his criticism at the union.”
Of course he did. Because that’s the (purely) political framing that is beneficial to reformers. They do this redirect all the time.
Greedy union thugs versus saintly reformers. They’ve been using this rhetorical tactic for a decade. It’s pure politics. There’s also a weird class element to it, as if working people are just a LITTLE distasteful.
http://en.wikipedia.org/wiki/William_Sanders_%28statistician%29
Dr. Ballou, in Lissitz (Ed.), 2005, “Value Added Models in Education: Theory and Applications,” analyzed the TVAAS and determined that value added-assessment of teachers are fallible estimates of teacher contribution to student learning, stating that standard errors of value-added estimates are large. Author thinks that value added models are merely one useful tool that should be used as one of many assessments in a comprehensive system of evaluation.
Researchers from the RAND corporation studied Dr. Sanders’ method and determined that his approach does not satisfactorily account for bias, cautioning that non-educational effects may be attributed by mistake to teachers, with no way of effectively determining the magnitude of the error.[2] Ballou (2002) and Kupermintz (2003) further support this claim, claiming that non-educational factors have a noticeable impact on the evaluation of teachers despite efforts to account for them in the model.[3]
The use of merit pay based on VAM has been discredited in articles by Dan Pink and more generally as a business practice in the Harvard Business review. The accuracy of VAM for evaluating individual teachers has been further discredited by the Economic Policy Institute and by mathematician John Ewing
Ohio’s VAM comes from SAS, where Wm. Sanders held forth until he retired. I tried to get detailed information on the formula being used and the other information about the contract with SAS.
I was informed that that the formula was proprietary. The three year contract for about five million was filled with SAS risk management strategies that shifts the blame for bad data and poor tests to the State of Ohio and its “providers” of the data and tests.
It is useful to remember that Lamar Alexander, former Governor of Tennessee and now in Congress, gave Sanders and his VAM “credibility” in education.
Also, Sanders applied his statistical methods to genetic engineering. From the get-go, the extension to education was intended to do a triage on teachers, stigmatizing poor breeders of test scores (teachers, schools) as inferior.
I believe that economist Eric Hanushek was among the first to introduce the concept of a value-added metric to rate a teacher’s production of gains in students’ test scores. He is among the most frequently cited researchers and proponents of VAM in federal and state policy circles. See Hanushek, E. (1971). Teacher characteristics and gains in student achievement: Estimation using micro data. The American Economic Review, 61(2), 280-288.
See also his reworking of “a Nation at Risk” rhetoric in Hanushek, E. A., Peterson, P. E., & Woessmann, L. (2013). Endangering Prosperity: A global view of the American school. Washington, DC: Brookings Institution Press.
Considering that it is completely UNETHICAL to use the results of any standardized test for anything other than what the test was designed to assess, WHY DO ANY TEACHERS TOLERATE THE VAM NONSENSE?
UNETHICAL to the hilt these VAM and SGP evaluation schemes are! UNETHICAL but legal because money buys votes.
Is it legal? Is there any precedents with VAM?
Near the beginning of this posting: “Tennessee is the state where VAM got started, launched in the 1980s by agricultural statistician William Sanders. ”
This is no exaggeration.
Read THE MISMEASURE OF EDUCATION by Jim Horn and Denise Wilburn (2013) for a detailed, if painful, account of the havoc that sprouted in and spread from the Land of VAM.
Take away the false edumetrics that go by such names as VAM and high-stakes standardized test scores and the edubullies have almost nothing left with which to club public education.
This is significant news.
The attorney who presented to the TN State Board pointed out an ethical breach in Huffman’s presentation. TN Commissioner Huffman showed a graph to assure the TN Board that a teacher’s TVASS score was stable over time. The attorney pointed out that the original source did not match Huffman’s graph, and showed that Huffman had removed significant information from the original. When the attorney looked on the TN Education website for the presentation , he found that this slide had been removed.
Why did Huffman misrepresent the source data and why did his dept of Ed remove the slide rather than defend it?
Reblogged this on peakmemory and commented:
More on the value added front
Reblogged this on Transparent Christina.
It’s about time that we teachers had some good news here in Tennessee. Kevin Huffman may think that eliminating or modifying VAM is a “small tweak,” but if he were a teacher he wouldn’t think it was little thing. This is just more evidence of how he looks down his nose at us.. Regarding the RTTT money- of course it was too much to turn down! Many teachers felt that it would have been a disservice to children to leave all that money lying on the table, so our union tried to negotiate the percentage that test scores would count in our evaluations. If we hadn’t found a way to accept the money, we would have looked like the bad-guy hold-outs who only cared about our own best interests.
Modify VAM? That’s like adding 2+2-=5. VAM is a factory model. Schools are NOT factories!
OK…I truly they (we ll know who) are trying to pull the wool over everyone’s eyes to keep them…..(the teachers) quiet!!!!
The only solution is to fire every teacher….then have elections…
Elected teachers will then be able to behave precisely as our present elected officials..Who knows..they may even get rich..read “Extortion”
There is no Power unless you are a Politician….or a Rich Company$$$$$$$$$
Far out….but I think true…
Things went from bad to worse here when Haslam and Huffman came along. Our DOE is a mess….a very intentional mess, funded in part by Mr. Gates.
http://www.gatesfoundation.org/How-We-Work/Quick-Links/Grants-Database/Grants/2011/09/OPP1041614
Because it’s critically important to distinguish between the methods of data analysis and the application of the methods, I offer a few points on the advantages of value-added modeling when used for the benefit of students and educators:
• TVAAS & EVAAS produce reports that are used by teachers and administrators to effectively design instruction across the full range of student performance, which is consistent with the goal of facilitating differentiated instruction;
• Training I received in the use of EVAAS (1) emphasized the instructional support the data provides to teachers and administrators, and (2) specifically warned against non-validated use of VAM data;
• Value-added modeling in TVAAS (Tennessee) & EVAAS (North Carolina) does not depend upon holding all other variables constant; rather it uses each student’s test history to account for contributing and confounding variables, which allows their effects on scores to be taken into account in the model rather than controlled in the usual sense of the word.