Audrey Amrein-Beardsley writes in her blog VAMboozled that the VAM-loving economists are giving each other high-fives for the triumph of VAM. Almost every state has adopted some version of it. Success!
What they don’t offer is any evidence that VAM has improved education.
Where are the districts or the states that have identified and fired the ineffective teachers and seen a vast improvement in their schools?
Why do the VAM ratings continue to bounce around? Why are teachers known to be successful in their school community getting low VAM ratings? Why are teachers often rated based on the scores of students they never taught?
Does reality matter?
Diane Ravitch's blog 
I love that question. “Does reality matter?” Lily Tomlin said, “No matter how cynical you get, you just can’t keep up.”
In case anyone is still thinking that economists are actual scholars or scientists rather than modern day witch doctors (and, frankly, given my choice, I’ll take the witch doctor).
“Economists are like psychics”
Economists are like psychics,
This cannot be denied.
Cuz if, by chance, they get it right,
It’s greatly AMPLIFIED!!
But mostly, they just get it wrong,
And utter not a word
For them to actually point this out
Would really be unheard.
And when their goof’s so blatant
They really can’t ignore it,
They simply claim they “found a flaw”
And “makeup will restore it”
SomeDAM Poet: the rapier rather than the broadsword—touché!— but…
In hopes of forestalling what is perhaps another inevitable deluge of angry emails, I would like to point out to protesting psychics and their support organizations that I was not the one comparing them to economists.
In all fairness, why cast aspersions on those employing mystical methods like gazing into crystal balls when VAM is fraught with such profound problems of reliability, validity and bias that they put the mentalist community to shame?
Of course, the above refers to Planet Reality. On Rheephorm Orb, where Rheeality Distortion Fields are the norm…
😎
P.S. Audrey Amrein-Beardsley, RETHINKING VALUE-ADDED MODELS IN EDUCATION: CRITICAL PERSPECTIVES ON TESTS AND ASSESSMENT-BASED ACCOUNTABILITY (2014). Shuts down Rheeality Distortion Fields real well.
“I reject that mind-set.” [Michelle Rhee]
Her chance of being wrong has gone from the 13th percentile to the 90th.
Really!
😏
What a FAB poem, SomeDAM Poet. I have to agree. 🙂
It’s a great big echo chamber they live in, so all they hear are themselves.
They also hear the cattle lowing
“A way for a Manager”
A way for a manager, a plan for his scam,
The little Economist laid down His sweet VAM.
The stars in the White House look down where He lies,
The little Economist, with powerful ties.
The cattle are lowing, Economist awakes,
And little Economist, a model He makes;
I love Thee, Economist, look down from the sky
And stay by thy cattle till morning is nigh.
For anyone who does not get the reference, VAM was originally developed to model cattle growth
Economists are self-appointed experts on everything, except fixing the economy.
When have they argued anything but an appeal to authority.
From the current New Yorker: ‘Lately, though, the annual performance review has been falling out of favor in some quarters. Microsoft and Gap are among several companies that have reformed their evaluation processes in recent years. On Tuesday, the consulting firm Accenture, an emblem of traditional corporate culture if ever there was one, announced that it is getting rid of annual evaluations for its three hundred and thirty thousand employees, replacing the process with a system where managers will give feedback on a more regular basis. Accenture’s C.E.O., Pierre Nanterme, told the Washington Post that the existing evaluations are cumbersome and expensive. Plus, he added, “the outcome is not great.”’
Does reality matter? As long as you do what I say, not what I do.
Why are economists so enthralled with VAM? The answer, I believe, lies in the economic benefits reaped by the ECONOMISTS. There is plenty of evidence refuting the effectiveness of VAM, but the economists ignore that evidence because it is not in their economic interests to acknowledge the failing of VAM.
“1984” has finally come to fruition.
We must get out before we are
VAMDamned!
Magic Elixir…NO EVIDENCE REQUIRED WHEN TALKING ABOUT LEARNING OR HOW THE BRAIN LEARNS…
Daniel Willingham talks about the tactics of charlatans who sell junk policies to schools, , and my essay about that phenomenon here http://www.opednews.com/articles/Magic-Elixir-No-Evidence-by-Susan-Lee-Schwartz-130312-433.html
Fivethirtyeight should be ashamed for advancing such dishonesty. They’re using the same strategy as the tobacco industry & the petroleum industry employed to discredit known scientific findings of the effects of smoking and fossil fuels, respectively. Read ‘Merchants of Doubt’ 2010 by Naomie Oreskes. These dishonest economists are no better than the climate deniers paid to cast doubt on the science of climate changes. The disinformation campaign began in the 1980’s and it’s taken 30 long years to break through the climate change deniers public relations noise. She called these men “Merchants of Doubt”
http://www.nytimes.com/2015/06/16/science/naomi-oreskes-a-lightning-rod-in-a-changing-climate.html?_r=0
“Her core discovery, made with a co-author, Erik M. Conway, was twofold. They reported that dubious tactics had been used over decades to cast doubt on scientific findings relating to subjects like acid rain, the ozone shield, tobacco smoke and climate change. And most surprisingly, in each case, the tactics were employed by the same group of people.
The central players were serious scientists who had major career triumphs during the Cold War, but in subsequent years apparently came to equate environmentalism with socialism, and government regulation with tyranny
The goal- cast doubt on the science.”
Don’t be surprised if you see this fake study pop up in litigation against the current spate of lawsuits against Value Added.
Shame on fivethrityeight.
FiveThirtyEight is Nate Silver’s site and though Silver did not write the article in question, Silver himself has a track record of weighing in on things about which he is woefully un- and/or mis-informed.
Climate Scientist Michael Mann took him to task for BS claims he made about climate change in his recent book “The Signal and the Noise”
FiveThirtyEight: The Number of Things Nate Silver Gets Wrong About Climate Change
Economist Steve Levitt (of U of Chicago), who also writes about “pop” econ (and “poop” science) was also taken to the cleaners for his own BS claims about climate change.
Some of these people are aware of their misinformation but others are just so far afield of their own expertise that they are simply clueless. Too many economists have a nasty habit of weighing in on things they know nothing about.
Here’s the “money quote” from Michael Mann
“When it came to areas like climate change well outside his own expertise, he [Silver] to some extent fell into the same “one trick pony” trap that was the downfall of Levitt (and arguably others like Malcolm Gladwell in The Tipping Point). That is, he repeatedly invokes the alluring, but fundamentally unsound, principle that simple ideas about forecasting and prediction from one field, like economics, can readily be appropriated and applied to completely different fields, without a solid grounding in the principles, assumptions, and methods of those fields. It just doesn’t work that way”
That applies to education as well as climate science.
STILL Bouncing around?
I love such questions. In my room, we called such insightful questions the EQ.
An essential question is special. The answers to your EQ s reveal THE PROCESS and THE HIDDEN TACTICS used to silence th voice of the practitioner of the discipline, and allow the charlatans to run public education… into the ground (while making oodles of money).
It is 16 years since the initial process, to remove teachers (which was put into place on NYC,) took me out at the top of a successful and celebrated career.
ANSWER: because the media has hidden it, and there is not a shred of accountability for the charlatans at the top who perpetrate the travesty.
You see, They needed a legal process, like VAM, one that did not involve criminal allegations!
They do not want to have to go through the process of creating criminal charges like then the one used to “get” Esquith (and me, and the tens of thousands of great, veteran teachers who whose reputations and to be destroyed in order to rid the system of their salary and their voice.)
http://www.perdaily.com/2015/07/lausd-blacklisting-teachers–how-they-do-it.html
http://www.opednews.com/Quicklink/LAUSD-OR-TARGETED-TEACHERS-in-Best_Web_OpEds-Deception_Evidence_Fired_Innocence-150720-360.html#comment555646
VAM made it unnecessary to go to such lengths to remove a teacher. like it did to David Packter http://protectportelos.org/the-david-pakter-saga-an-all-too-familiar-of-a-story/
Yes, the ‘gotcha’ squad is still at work, http://nycrubberroomreporter.blogspot.com/2009/03/gotcha-squad-and-new-york-city-rubber.html
see the case of Francesco Portelos.
http://protectportelos.org/allegations-against-me/
But, VAM makes it so easy to remove teachers before they are vested, but having VAM did not stop THE real PROCESS– their tactics of civil rights abuse.
http://www.perdaily.com/2011/01/lausd-et-al-a-national-scandal-of-enormous-proportions-by-susan-lee-schwartz-part-1.html
Successful marketing does not make ones product a success. The cigarette industry was very successful in getting millions to believe that smoking was cool and sexy. Doctors once got everyone to believe that bloodletting was the cure for most diseases. The Roman Catholic Church even convinced most of their believers that a monetary contribution reduced the number of years spent in purgatory. These economists should ponder the consequences of these three examples as they pat each other on the back.
OF COURSE VAM is successful – at putting veteran teachers in rubber rooms, into early retirement, and out of the teaching profession, to make way for those younger, smarter, oh so inexperienced and questionably cheaper TFAs. Succe$$. This is but one part of the multi-armed Octopus that is rheeform. Coming soon to a district near you….VAM, TFA, and a Broadie.
“In an article title ‘The Science Of Grading Teachers Gets High Marks,’”
It shouldn’t take very long (a second maybe) after reading the title of the article AA-B was commenting on to realize just how idiologically (purposely misspelled) driven the authors are. “Science of Grading Teachers” Yeah, right, and I’ve got some great ocean front property for sale cheaply over at Lake of the Ozarks in Central Missouri for those authors.
Economists love to get that word “science” in at every opportunity.
They are very self-conscious of the fact that those in other fields do not consider them scientists and manifest their ‘science envy” with excessive mathturbation and emphasis on “sciency” terminology — stealing terms from fields like physics and applying them in completely new and totally nonsensical ways.
They don’t even seem to realize how foolish it makes them look.
As William Black has said, “Economics could be a science if more economists were scientists”
I know you’ve read that, but in case others have not, here’s the link
The following quote from the William Black article basically says it all:
Chetty has conflated “data” with “facts.” Accounting control fraud produces fraudulent accounting data that economists and finance scholars treat (implicitly) as facts. During the expansion phase of a bubble or an expanding epidemic of accounting control fraud the traditional econometric study will lead the “scientific” economist to support the worst possible policies that most aid accounting control fraud. Whatever practices best facilitate the creation of fictional income will show the highest positive correlation with the firm’s reported income (or stock price). The true (negative) “sign” of the correlation will only emerge after the bubble bursts or years after the fraud epidemic begins. By then, of course, it is too late to prevent the crisis brought on by the “scientific” economists’ disastrously bad policy recommendations.”
//end of William Black quote
Black’s observation has direct relevance to the use of VAM because the entire argument of Chetty and other VAMbots rests on the assumption that the data accurately and adequately reflect “student learning” and that they are valid for use in assessing teacher effectiveness.
Conclusions based on invalid (or unvalidated) data and/or assumptions are worthless.
What Chetty and the others are doing basically amounts to mathturbation: meaningless mathematical diddling.
“The VAMbots”
Mathturbating
Data diddlers
Teacher rating
Mathy meddlers
Late entry.
What has enabled the abuse of VAM?
1. A political environment marked by a distain for educators including educational researchers.
2. A long delay in any statement from the American Statistical Society on the misuse of VAM for evaluating individual teachers.
3. As Audrey Amrein-Beardsley points out (as have others) the marketing of VAM under the softer rhetoric of “measuring growth in student learning” and the repurposing of those same measures” for judging “effective” teachers—a measure reduced to the production of year-to-year gains in test scores of their students—known to be invalid.
The less publicized story is the explosion and standardization of “big data” in education about a decade ago in the midst of dealing with the requirements of NCLB. That standardization has made number crunching much easier. Consider these strategically important events.
In 2005, The Bill and Melinda Gates Foundation invested $75 million in “The Data Quality Campaign” a major marketing effort to standardize data gathering, aided by the National Governor’s Association, the Council of Chief State School Officers, Achieve, and The Education Trust—the same groups pushing the Common Core. During the same period and until 2011, the Gates Foundation also awarded grants totaling $390,493,545 for projects to gather data and build systems for reporting on teacher effectiveness.
This multi-faceted campaign helped states create the Teacher Student Data Link (TSDL) system, intended to serve eight purposes: 1. Determine which teachers help students become college-ready and successful, 2. Determine characteristics of effective educators, 3. Identify programs that prepare highly qualified and effective teachers, 4. Assess the value of non-traditional teacher preparation programs, 5. Evaluate professional development programs, 6. Determine variables that help or hinder student learning, 7. Plan effective assistance for teachers early in their career, and 8. Inform policy makers of best value practices, including compensation (TSDL, 2011, “Use and Purpose).
One of the most important functions of the TSDL system was to monitor the work of teachers in a manner that ensured all courses were based on standards, and that all responsibilities for learning were assigned to one or more “teachers of record” in charge of a student or class. A teacher of record is best understood as person who has a unique identifier (think barcode) for an entire career in teaching. A record is generated whenever a teacher of record has some specified proportion of responsibility for a student’s learning activities …and especially the associated test scores of those student on state wide tests.
The TSDL system enabled states to create and monitor current and longitudinal data on the performance of teachers and individual students, as well schools, districts, and states. The system is actually intended to reach into teacher education programs, specific courses, faculty, and job placements of graduates. VAM for teacher education faculty is being envisioned— monitoring the test scores of students taught by graduates of teacher education program.
The Data Quality Campaign described the ultimate aims: “to determine the ‘best value’ investments in education and to monitor improvements in outcomes, taking into account as many demographic factors as possible, including health records for preschoolers.”
The Gates-funded TSDL campaign added resources to a parallel federal initiative. Starting in 2006, the U.S. Department of Education invested over $700 million in the Statewide Longitudinal Data Systems (SLDS) Grant Program. More than forty states received multi-year grants to standardize data on education. Operated by the Institute of Education Sciences, the SLDS program was: “designed to aid state education agencies in developing and implementing longitudinal data systems. These systems are intended to enhance the ability of States to efficiently and accurately manage, analyze, and use education data, including individual student records…to help States, districts, schools, and teachers make data-driven decisions to improve student learning, as well as facilitate research to increase student achievement and close achievement gaps.”
The standardization of data for education has accelerated and expanded since 1990, making it easier for statisticians to crunch numbers, deal with missing data, and grease the whole machinery of VAM.
Along the way, few people understood that federal requirements for “growth measures” were asking for the use of VAM, while leaving the specific algorithms up to states.
Standardizing testing has been the great enabler of the VAM Scam.
The aggrandizement of VAM in education has had many spillover effects. VAM helped to create the absurd category of “teachers of untested subjects.” VAM has enabled the marketing of invalid strategies for evaluating about 69% of the teaching workforce who do not produce data for VAM. For those teachers USDE has marketed the equally process of meeting Student Learning Objectives, as if SLOs are potentially comparable to VAM.
The focus on VAM has totally distorted the claims about American education in the press and in policy formation.
It has truncated peer-reviewed empirical educational research. We are awash in studies based only on test scores in reading and math. Too often these studies spawn claims about the efficacy of this or that intervention, or professional development scheme—as if the “findings” from test scores in two subjects are valid and relevant to teaching in every subject and grade. Among other consequences, teachers endure hours of “trainings” keyed to a narrow view of education, and the whole curriculum becomes narrowly focused a 21st century version of the 3 R’s.
Economists and statisticians, aided by test makers, have helped politicians misrepresent the status American public education, the quality and dedication of teachers, and the accomplishments of students.
I am grateful for all scholars, economists, statisticians, parents, students, and taxpayers and holders of political office who have a more informed and ample view, especially those who are stepping up to criticize and reverse the flawed and failed policies of the last several decades.
This is wonderful. Thank you. Has anyone done equivalent analyses of the gender and racial implications of VAM?
Paul, contact Audrey Amrein-Beardsley.
Audrey.Beardsley@asu.edu
Thanks to the Keynesian economic policies of the Fed. since Greenspan, we are now $18 trillion in debt, and the American dream for our children is over. Every newspaper in the country prints articles by the ultraliberal Paul Krugman, who has the brains of a ‘pickled herring.’ In order to sustain their social ideology, the Fed. instituted a spending policy and rationalized it with unproven claims and lies. Remember “for every $1 given away to assist the indigent, the economy reaps $2 in benefit.” Nonsense. You can’t call yourself an economist without having a theory of your own, so they all write a theoretical dissertation, publish a very expensive book that is requisite for anyone who desires a degree in economics, and then set out to prove it, unfortunately at the expense of the taxpayer and the country. The economist Kenneth Galbraith said “The only function of economic forecasting is to make astrology look respectable.” Jobs, jobs, give the minorities jobs. No, I say set them up in business, and force our mega wealthy oligarchs to share the wealth and lend their expertise to insure successful enterprises. Let them divy out the jobs. Legislate the breakup of the large ‘box’ stores that have destroyed the growth of small business in minority areas, and reinforce family life in those areas by giving incentive to family owned enterprises. We can’t continue to allow the wealth of the nation to accumulate in fewer and fewer hands. That’s how capitalism should be working.
Ian Kay
Obviously, one of the reasons children are given standardized tests is so the scores may be used for teacher ranking. However, this is barely mentioned when parents are scolded or lectured on why children have to be tested. Instead we’re told testing is necessary for equity, or testing is just like vaccinations, or testing is just part of life.
It’s an odd omission. I didn’t hear a word about it from the US Senators who were pushing testing in the NCLB reauthorize update although pushing states to adopt teacher rankings based on student test scores was a huge part of the Obama Administration’s RttT. Without annual standardized tests there is no VAM. True, right?
Why is VAM omitted in the defenses of standardized testing? How big a role does VAM actually play when lawmakers are mandating annual testing? It must play some role. Federal and state government spent a bundle on these systems and a lot of prestigious people backed the ranking schemes.
Fully expected. Controversial free market economist, Steven Levitt’s Freakonomics website, had a fawning piece about Nate Silver, of FiveThirtyEight, in April.
Levit’s site has a posted article, “Is it O.K. for a Restaurant to Racially Profile Their Employees?”, one about access to Medicaid resulting in more smoking by poor pregnant women, and, one about lack of innovation, the latest myth being developed, as an indictment of higher education.
Levitt, affiliated with the University of Chicago, is on a charter school board with Harvard Kennedy School faculty.
I’ve concluded Freakonomics’ blather is a harbinger of, or aligned with, plutocratic talking points.
New commenters at Freakonomics, often seem surprised that their inference, from the sites’ title, is wrong. The title suggests an iconoclast’s opinion. IMO, the site’s posts reflect free market dogma, masked with wildly speculative “research”.
Nate Silver gained credibility for an accurate electoral prediction.
It’s illogical to credit a broad spectrum of a person’s work, as objective, based on a thin thread.
Paul Krugman, an economist who verbally sparred with Silver,
has shown the premises of the policy makers, often reflect dogma with flawed data, for rationale.
https://dmaxmj.wordpress.com/2015/07/26/the-selective-and-convenient-use-of-vam/
Stop pretending…if you’re going to do the VAM, then REALLY do it.
Dear Laura,
So wonderful, erudite and true.
My contribution to his conversation and to this site is a knowledge of the PROCESS THAT PRECEDED VAM! If the lack support, and the genuine lawless harassment does not send a teacher out the door before it is time to vest, then they can use VAM and any subjective documentation to prove incompetence.
I was there, to experience it…. to see the process begin,– and how it was used to remove the veteran core of teachers in NYC in themes heinous despicable way.
I Posted this many times here, so forgive me if you have already read it, because this was PRE-VAM although in LAUSD, it is still the go-to tactic.
I wrote it in 2004, but Perdaily put it up in 2011 because it does not open completely at my site.
http://www.perdaily.com/2011/01/lausd-et-al-a-national-scandal-of-enormous-proportions-by-susan-lee-schwartz-part-1.html
This particular piece shows that what they did to NYC teachers, and to me, is still a tactic in LAUSD (no surprise to us in NYC when our old& failed chancellor Cortines popped up there)
http://citywatchla.com/8box-left/6666-lausd-and-utla-complicity-kills-collective-bargaining-and-civil-rights-for-la-s-teachers
I try at this blog, to explain to all newer hires, and those facing the evaluation swamp, how VAM was NOT the first step, but the NEXT step they used to remove the expensive veteran one that prevents any teacher-practitioner from staying more that a few years.
In medicine or science , or in engineering, research must be accomplished on a LEVEL that demonstrates it will work EVERYWHERE is the rubric. It just cannot work in one place and not in another.
Only in pedagogy –the profession that must grasp how the brain acquires both skills& knowledge– is no evidence necessary, and any cockamamie curriculum or program be mandated based on nothing. http://www.opednews.com/articles/Magic-Elixir-No-Evidence-by-Susan-Lee-Schwartz-130312-433.html
IN all complex disciplines, the average person needs someone to explain what works and what does not. The citizens in this nation do not find out about what LEARNING requires, because the topic is TEACHING & evaluating TEACHERS… whites easy to do when you get to invent the rubric and the criteria, and the test.
Pundits and experts abound, all of them in thrall to the oligarchs who need the voice of the genuine educator GONE! This tool to remove the teacher-PRACTITIONERS, foes not merely make doodle of money for privateers, it allows the legislatures to take over the nation schools and dictate/mandate what the kids will hear. They can change history and what our kids know about science, and what our future citizens who are not kids for long, actually can do.
This is how democracy has been vanquished. VAN did it, but so did the end of six amendment rights for those Americans who just happened to choose teaching, and the education workplace as their career.
Saudi Arabi knows how this works… they have the ‘midrash’, which gets them early and tells them their version of EVERYTHING.
This was my comment, as a lowly parent & taxpayer, at the 538 article comment thread:
As a taxpayer and parent I’d like to know what this article has to do with economics. I can find nothing in the article, nor in the comments thread, which speaks to (much less ‘proves’) that VAM shows any results which save the taxpayer money. As a member of a community I am well aware of the time our teachers have been spending for the last two years on the voluminous paperwork required to implement VAM. Although they seem to be doing it without extra pay, this is worrisome. These people already spend too many hours on a medium-salary job, and with the extra stress, that will push more of them out of a difficult profession.
What’s worse at least in my area (NJ), VAM seems to be a 2-pronged affair:
there’s the SLO/ Danielson or Marzano business that has them adding short autumn and EOY assessments in every subject to measure progress — which sounds like debunked & easily-gamed MBO junk
I had to do in the ’80’s private sector–
and then we have the students’ PARCC scores which will soon become a weighted part of the teachers’ annual evaluation.
The SLO business adds a couple of hrs’ testing in every subject (so 10 hrs of class time per year)– minimal yet to my mind from experience a waste of time.
The latter– PARCC CCSS-aligned tests, taken on computer, disrupted the schedule horribly this year. Just to administer them required weeks during March & May to shuffle students by group in & out of gym/ library/ computer lab, where available laptops were corralled. That meant those areas were lost to normal physed, library research projects, curriculum-associated computer-lab work, for 3 – 4 wks of the 12-wk spring semester.
Much class time was spent teaching primary students how to keyboard, drag-&-drop, scroll, not hit buttons that would freeze the screen. And much more class time was spent drillimg kids on the PARCC Q&A format.
Although the PARCC tests are administered solely to collect data on school & teacher performance (meaning by the way older students recognize they have no ‘skin in the game’)– because they are or soon will be hi-stakes for school & staff, much time & attention is paid.
Did I mention my district has long been one of the highest-performing in the state? Here in NJ rich districts like ours send a large chunk of our RE taxes to poor districts; we pay 96% of the school budget ourselves. And now we must sacrifice many weeks of our precious, highly-paid-for curriculum-learning time to… VAM???
OK now I’m mad. I can read right on this board that there’s absolutely no proof that any of this supposedly economically-driven activity has anything to do with lowering my property taxes. In fact, what I can glean by reading the national news, its primary effect to date has been to drive veteran teachers into early retirement, to be replaced by substitutes, TFAs, & newbies (which is what I see is happening in Newark, where much of my RE taxes go, which is why mine are cripplingly high; as far as I can determine costs are higher than ever in fact Zuckerberg’s $100million went to consultants and state administrators).