The centerpiece–and the most destructive element–of Race to the Top is the insistence that teachers must be evaluated to a significant degree by the test scores of their students, whether they go up or down.
It is destructive because it makes standardized tests the purpose of education.
The tests cease to be a measure and become the aim.
That is wrong.
It leads to a narrowed curriculum, teaching to the test, and cheating.
And the measure itself is fraught with error. The teachers with high ratings one year may get low ratings the next year. Some with low ratings may get high ratings the next year. They did exactly the same things but their ratings shifted. One gets a bonus, the other gets fired. It is wrong to make the tests so consequential.
Here, if you have not read it, is an excellent summary of the VAM research, explaining why VAM is misused, by the distinguished psychometrician Edward Haertel, presented in a lecture to ETS.
You should also follow VAMboozled, which is testing expert Audrey Amrein-Beardsley’s blog. She will publish a book this spring, showing the invalidity of VAM. She points out that more than 90% of researchers in the related field agree that VAM is misused by federal policymakers.
Children are not data points; teachers do more than tests measure. Education is more than standardized tests can measure.
Neither children nor teachers nor education itself can be reduced to a metric or an algorithm.
VAM is Junk Science.
In a rational world, NCLB and the Race to the Top would be consigned–quickly–to the ash heap of history.
Hang on, friends. That day is coming.
Diane Ravitch's blog 
VAM is Junk Science.
The principal stated that the DOE told her she could not give a 5 unless the teacher taught teachers….
I was a mentor to every new teacher that walked in..and teachers used to all work together….
The principal knew that but had to follow the DOE’s requirements..
You must present evidence..I had a flash drive that could store a library and I brought up the evidence….one piece at a time…..
The evaluation is a piece of cr*p for lack of a better word…it does not deserve a better word….
Teachers DO NOT work together anymore..It is a Dog eat Dog situation..Period..
Only a teacher can choose to abdicate their collegiality.
I’m not too sure I can hang on much longer. The new CT teacher evaluation process borders on the absurd. I spent almost six hours writing the pre observation form and almost three hours completing the post observation form: all this for a one-hour lesson! Guess what didn’t get done that week: lesson plans, correcting, classroom preparation, and contacting parents. How does any of this nonsense (out of deference to you, Diane, I’ll hold off on the expletives that come to mind) make me a better teacher? I spent part of last week trying to figure out how early I can afford to retire. I would be very stressed out right now if I were a teacher beginning his or her career.
“borders”???????????
Ah, the lawyers. Working for them, too.
We have monetized evaluation.
I stand corrected: it is absurd!
Can you say share if you are CEA or AFT? CEA has organized meetings with legislators this week. I suspect it is to pretend to listen and then get new talking points for Dannel.
Many teachers in CT are losing their minds and figuring out when and how to get out.
What you describe is exactly correct. I spent two weekends writing two SLO’s and now they tell us we have to change our IAGD and with no guidance or support. Just figure it out yourself.
The administration had the “training”. We had the drive by and we are on our own. They really don’t know what they are doing.
It is survival of the fittest. We have several people out sick and everyone is stressed out. Great environment for the kids, eh?
I did not make it to the local CEA meeting because I had seven meetings that week, most during my planning periods. I will contact my state representative, a Democrat. I gave already spoken to him while he was campaigning door-to-door. I assured him that Malloy has a snowball’s chance in hell of getting my vote come reelection time.
VAM is actually highly effective – in doing what the reformers pushing it intended. It’s dismantling the teaching profession. It’s building the case for employing cheaper teachers.
That the ratings are unreliable and even not valid is irrelevant or even a positive thing to those who who would dismantle then privatize public education.
VAM is working. Quite well.
The bad news for the reformers is that we now have a viable target.
Absolutely! VAM is one gigantic bulls eye.
VAM and standardized testing are the result of the Campbell’s Law effect being weaponized and used deliberately for monetary gain.
Campbell’s Law (FYI)
Campbell’s law is an adage developed by Donald T. Campbell:[1]
“The more any quantitative social indicator (or even some qualitative indicator) is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.”
The social science principle of Campbell’s law is sometimes used to point out the negative consequences of high-stakes testing in U.S. classrooms.[2]
What Campbell also states in this principle is that “achievement tests may well be valuable indicators of general school achievement under conditions of normal teaching aimed at general competence. But when test scores become the goal of the teaching process, they both lose their value as indicators of educational status and distort the educational process in undesirable ways. (Similar biases of course surround the use of objective tests in courses or as entrance examinations.)”[1]
Bet Duncan has no clue about Campbell’s Law. He’s a third string basketball player and we have thrid string politicians who work for the rich…wh—r–.
Peter Smyth: you are not the first to point out that a proven ‘worst business practice’ is being used against teachers, students and communities.
A proven expert in management with extensive experience and academic credentials, as well as an outstanding numbers/stats background, put it thus when discussing “Numerical Goals”:
[start quote]
A numerical goal is a number drawn out of the sky. … A numerical goal accomplishes nothing. What counts is by what method. … If you can accomplish a goal without a method, then why weren’t you doing it last year? There’s only one possible answer: you were goofing off. May the numerical goals be achieved? Yes. We can make almost anything happen. But what about the cost? What about the loss? Anybody can achieve almost anything by distortion and faking, redefinition of terms, running up costs.
[end quote] [W. Edward Deming, THE ESSENTIAL DEMING (2013, p. 55)]
And although Deming said the above in 1992, he was saying the same thing even before Campbell’s Law was formulated in 1975.
Alan puts it succinctly: “VAM and standardized testing are the result of the Campbell’s Law effect being weaponized and used deliberately for monetary gain.”
Yet again, the leading charterites/privatizers flaunt their blind adherence to Marxist doctrine:
“Politics is the art of looking for trouble, finding it everywhere, diagnosing it incorrectly and applying the wrong remedies.”
¿? Groucho.
Of course…
😎
Try not to be intimated. VAM is unreliable, invalid, and will do harm.
There is a long line of lawyers drooling over as we post.
. . . over THIS as we post
Agree and agree. But I am curious. How many of the posters here with school-aged kids of their own.have refused to allow their children to participate in the testing? Civil disobedience will prove to be the most effective tool. If every teacher refused to allow their own child to take a high stakes test then the system would begin to come crashing down.
I have. My children loved it, and my oldest refused to take an “ACT preparation test” on his own. I am now telling teachers this same thing. Most have no idea, and neither did I until about a year ago. Even my state legislator did not know he could opt his children out.
I have, he responds to the principal and debates him as to why he should take the test. The principal stated that the test was to help his teacher. He replied, she knows me well and already knows what I can and can not do. I am not taking this test, I like my teacher, she should not be punished over a test I am taking. His annual finishing comment, he writes it on the test form, “My name is Nate, I am not a test score.”
These thoughts and insights need to be printed in a larger font:
“Children are not data points; teachers do more than tests measure. Education is more than standardized tests can measure.”
“Neither children nor teachers nor education itself can be reduced to a metric or an algorithm.”
and distributed to an ever-increasing readership. They will become those quotes cited and copied repeatedly to refute the destruction of public education as it is replaced with a corporate model in which the public will lack any input, influence or control.
Too many veteran teachers are being destroyed over this. We have worked our entire lives for the kids and now being told you are a this or that. I say this entire realm of education needs an overhauling. I think all better start rethinking what life is all about and what we truly want for the future for our kids. When I hear top notch teachers now saying, “Has everything I have devoted the last 30 years to been worth it?” then I say we have trouble here. It will take a miracle to fix it.
This is a great point! Last year I lost two brothers and my son had serious medical issues. Needless to say, I’m sure my test scores were extremely low but for the first time ever, I never had time or energy to analyze the scores!
After all, aren’t teachers human?
We also have so many tests with errors, computer equipment issues, lack of computers for tests to be completed by all students at the same time, and other climate/student conditions which essentially makes the testing invalid.
Reblogged this on peakmemory and commented:
Note the link to the VAMboozled blog: http://vamboozled.com/
How would the National Dental Association have responded if the federal government asked their leaders to sign off on a policy that rated the effectiveness of their members (dentists) based on the number of cavities in people who were not their patients?
That should be a rhetorical question based on the pure absurdity of the idea.
How did our union leaders respond?
Show us the money!
Although the slogan is “VAM SHAM,” I propose another variation: It’s a VAM Shame!
VAMs don’t capture growth. I know proponents state that it does, but that’s not an accurate claim. The text/passages used in high-stakes, standardized tests are “on grade-level.” So, if a student, who’s multiple grade levels behind, fails to access the content, then his or her score will inevitably suffer. Even though this student’s score appears low, he or she may actually have gained meaningful growth. I’ve explained a couple scenarios, in more detail here: http://ward8teacher.wordpress.com/2013/09/22/vam-design-flaw-where-you-teach-matters/
As a public middle school teacher, who works within a high-poverty middle school, I can assure you that a majority of our students are well behind middle school level competencies. So, as long as VAM is used to measure teacher “effectiveness,” there’s little incentive for teachers to work in schools that need the most. This is a VAM Shame!
One more important point that supports our “invalid claim hypothesis”
Our evidence is overwhelming.
The burden of proof should be on the reformers. They claim that VAM
is valid method for measuring teacher effectiveness, yet they have produced zero evidence to support this absurd idea. For a group of so-called experts that genuflect on the data alter, this absence of quantitative evidence in support of their claim should tell us (and any attorney) all we need to know.
A sham and a shame indeed. And shame on them for imposing it. And shame on the AFT and NEA for endorsing it.
This year, after years of teaching “average” or “grade level” students (many of whom were actually a few years below grade level), two-thirds of my students are “honors”. I have no behavior issues in these classes. I have motivated, eager learners who enjoy and value learning, most of whom have strong parental support.
This is in stark contrast to what I have been accustomed.
This also gives me a strong advantage over my colleage across the hall, who teaches LRE science. The majority of her students have serious behavior issues, and many have significant cognitive issues.
How can we be fairly evaluted with such a system?
DOE doesn’t want to hear it. The Secretary of Ed ( recently appointed to Chiefs for Change) doesn’t want to hear it, and our governor denies this issue as well.
It’s as if we are pawns in a Great Fraud on the public.
Amen…pawns in a Great Fraud on the public is right.
DEMAND that they provide evidence that VAM id valid.
VAM aint VALID.
A grammatically lousy slogan for a teachers but it has decent alliteration.
VAM’s not VALID
and we can prove it.
When considering a new drug, the FDA asks two questions, the answers to which determine whether the drug goes to market:
1. Is the drug effective in treating the intended condition? “Effective” of course is relative.
2. Does the drug do harm through significant side effects?
Note that patients and healthcare professionals have access to this information.
Note also that recently questions have been raised about the credibility of reporting clinical trials.
But ed reformers should be required to answer the above questions.
Note that they have actually done the tests before a drug goes to market. In the case of educational policy, the public is the lab rats.
Keep pushing everyone and the correct change will happen. What is missing, however, is a viable alternative. As I have said a zillion times for the last 20 years, if their is no alternative plan, public schools will, and should perish. Go here for some ideas and then make the plan your own http://www.wholechildreform.com.
This plan was field tested at the Milwaukee Village School from 1995 – 98 and is documented in our books at that website. we went through living hell to get it started and keep it going as documented in our “Quashing the Rhetoric of Reform” book. And came to our conclusions in our “Saving Students From A Shattered System” book.
These are real comments about the things we did as well as the things we wanted to do. We had the same obstacles then as exists today. And have the battle scars to prove it.
Now, almost 20 years later, the time has come. The testing fiasco is crumbling. The only thing keeping it alive is the lack of a viable alternative.
Let´s get at it. How can I help?
I respectfully disagree with your assertion that VAM is junk science. Please don’t confuse junk policy with junk science.
David Cooper,
The overwhelming body of research shows there is no science behind VAM. They say it is inaccurate, unstable, unreliable. Yet VAM is used to fire people and destroy their lives. It is junk science.
Dear Diane,
I just re-read Haertel’s paper, and it confirms my original position: it’s not VAM per se that is junk, it’s the misuse of VAM in state and federal policies that we can all agree is junk. When properly applied to teachers’ planning for differentiated instruction, VAM is a useful tool, not a panacea. We owe the scientists who work in this area the respect afforded to any professional who attacks a tough problem in good faith, guided by sound theory, and who puts their methods and findings into the public domain for peer-review and popular scrutiny.
David, I have a deeply principled view that teaching cannot be reduced to a number. What is the added value of a philosophy course? History of art? How do you measure love of learning? Teaching is art and craft, not science. As I read Haertel, he said that VAM scores should never be a set percentage of any teacher’s evaluation. I agree. The principal should note whether scores go up or down, but take into account which children are in the class. If they are all profoundly disabled, there may be little or no test score gain. The tests should be used to help, not to punish or reward.
Lets not forget that states adopting RTTT/NCLB waivers were required to evaluate all teachers using standardized test scores.
However CCSS tests are only administered in math and ELA in grades 3 to 8. The majority of teachers have no CCSS aligned test with which to be measured. This fact alone will get VAM tossed in any court of law.
Diane
I too respectfully disagree with your use of the phrase “junk science”
Its an insult to all junk and especially insulting to bad science.
“In a rational world, NCLB and the Race to the Top would be consigned–quickly–to the ash heap of history.Hang on, friends. That day is coming. ”
Diane, from your keypad to god’s ears. Only…. What’s taking sooooooo long?
Deborah
Only NY and KY have felt the full brunt of CC testing and VAM evaluations. Just wait until this slow motion train wreck affects over 20 million 8 to 14 year olds this year. It will multiply the parental outrage seen on Long Island, in Westchester, Buffalo, and many other regions of NY by 40 fold. Some cracks are starting to show as political pressures are giving many state legislators second thoughts. And a cavalry of lawyers is not far behind.
Having lived through the first round in NYS, and being a not insignificant part of the outrage on Long Island, my heart goes out to all those little kids. Our legislators need to step up to the plate and soon before more little spirits get crushed.
Call your reps. Call Cuomo.
Once Andy’s poll numbers start to dive on this issue, he will act.
Have been calling and emailing, including legislators and Regents. Cuomo’s approval rating already at an all-time low.
I am expecting them to approve a 3 year moratorium. If so, CC will never be seen again in NY. We’re one big domino.
Well then, from YOUR keyboard to god’s ears! Hope they act soon. Test prep and testing season is upon is. Now if we could just regain control of our children’s data….. http://petitions.moveon.org/sign/protect-new-york-state
New acronym for VAM = Void And Malfunctioned
Diane and others concerned about VAMs.
It is important to remember that about 70% of teachers do not have job assignments that generate the state-wide test scores that VAM munch and crunch. The equally abusive proxy for these teachers is known as a Student Learning Objective (SLO) or Student Growth Objective (SGO). SLOs and SGOs have the same bottom line. They are a derived from Peter Drucker’s The Practice of Management (1954) and old-school behavioral objectives. The version in many schools comes from a group in charge of implementing the Race to the Top of agenda. See http://www2.ed.gov/about/inits/ed/implementation-support-unit/tech-assist/slo-toolkit.pdf
Teachers are required to write one or more SLOs/SGOs for their classes, typically using a computer template. A trained external evaluator assigns a grade to each SLO/SGO using a checklist of about 26 criteria in eight categories: Rationale, Population, Interval of time, Assessments, Expected growth, Learning Content, Teaching Strategies. Evaluators are not usually principals because they may be evaluated on the quality of the SLOs/SGOs of their teachers.
Teachers must analyze baseline test data on students (prior year tests, pretests) and set “targets” for pre-to-posttest gains in scores, The tests must to be state or district approved for use with all students and subgroups enrolled in the course/grade level. Teacher’s must list their teaching strategies and cite research or other evidence for their “targets” and instructional strategies..
A “trained” external evaluator rates the SLO/SGO using a four or five point scale—“high quality” to “unacceptable” or “incomplete.” The rating may also be planned to show the gains in scores that students must achieve for the teacher to be judged “ highly effective,” “effective,” and so on. Later in the year, teachers who have similar district-approved SLOs are rated and stack ranked on the gains in test scores they have produced.
Bottom line: “Growth” is a euphemistic name for a gain in test scores from one point in time to another. Meeting “growth targets for learning” is like meeting a sales target or a production quota by a date certain. Students are performing “on grade level” if their test scores are at or above the median on a percentile scale (1-99). A student is said to have achieved “a year’s worth of growth” if his or her gain-score on a proficiency test is equal to, or greater than, the gain-score made by a 50th percentile student. Teachers in some districts are rated “highly effective” only if all or most of their students have gain-scores of “more than a year’s worth of growth.”
A 2013 review of research bearing on SLO/SGOs documented unresolved issues in the validity, reliability, efficiency, and fairness of these measures for high-stakes evaluations of teachers in a wide range of subjects and job assignments Gill, B., Bruch, J., & Booker, K. (2013). Using alternative student growth measures for evaluating teacher performance: What the literature says. (REL 2013–002). U.S. Department of Education. Retrieved from http://ies.ed.gov/ncee/edlabs
Another USDE policy for teacher evaluation that qualifies for a Ravitch award as junk science.
I don’t know of a single administrator who would even read that.