A reader points out that value-added assessment is fundamentally flawed. It is unscientific and lacks validity and reliability. It will narrow the curriculum and promote cheating. Yet Race to the Top has pushed it, Secretary Duncan lauds it, and almost everyone (except education researchers) thinks it must be right, because…everyone is doing it. But it’s wrong. No other nation is doing it.
VAM continues to strike me as unscientific. As a science teacher I am continually astounded by the lack of scientific integrity when these methods are employed to personally, and professionally, evaluate teachers.
The validity of such methods are the first red flag. VAM uses the philosophy of economics which should be a crushing blow right off as economics is as soft a science as they come with a myriad of “lurking variables” ready at any given time to destroy the ability to measure any one variable, i.e. teacher effect. I cannot believe more science and math teachers, alongside mathematicians (I know Ewing stepped up) and scientists, have not come out in disgust over these methods. Of course, they have somewhat, but their voices cannot overcome billions of dollars acting as carrots to states under RttT and ESEA flex to adopt these policies.
The reliability of VAM, which has come under fire, really is moot. If a process is not valid, then questions of reliability are meaningless. I laugh when I read that VAM is unreliable, because real scientists know that if a process is invalid, reliability measures are meaningless. Economists are not real scientists.
If VAM were used as a diagnostic, then it would be acceptable, but it cannot be used as a diagnostic as its only purpose is to employ punitive measures. The VAM empire knows this, including SAS. VAM is not designed to help kids learn – it is designed to fire teachers under its own invalid measures measured solely on test scores.
The “multiple measures” of teacher evaluation are meaningless. Administrators will be biased concerning a teacher’s past VAM scores and will be scolded for not matching scores under the premise that they are not good evaluators (this is already being documented in HISD and in TN). Now, we have just placed more power in the hands of administrators to relieve themselves of whatever teachers they wish.
What good is a “projected score” to teachers? Teachers can do nothing to remedy a student who may not make “growth”, because there is no way to know who will or who will not make growth. No teacher-made assessment gives teachers a clue as to what students will or will not grow. A teacher may have a students that earns an A+ on all their assignments from the teacher and may still not show growth because they may have also earned A+’s from their previous teacher also. It used to be, at least with “proficiency”, that a teacher could figure out what students needed help and zero in on those students in order to attempt to get them to reach proficiency, but what are teachers to do with “growth”? How do teachers measure whether a student will or will not show growth? It is an impossible target.
I believe that target was made impossible for a reason. Just as NCLB was designed to show schools as a failure, so too, in 10 years, will reformers look back at NAEP, PISA, and TIMSS outcomes, and declare that teachers are failures.
You can bet that state mandated tests will increase as teachers rush to teach to the test, teach THE test (for those who can see or get their hands on them), or downright cheat.
May the best test prep. teacher or cheater win and hold their job and barely feed their families to do what they love. Good luck.

Or, give the first test of the year on Friday afternoon, or the first day back from a long weekend. Be sure to announce the time every 20 minutes.
LikeLike
Diane,
VAM troubles me a great deal. Perhaps it is the false promise inherent within VAM models that disturbs me the most—-that is, an instrument sold as a way to bring ‘absolute objectivity’ into teacher evaluations is no more object or accurate than flipping a coin to determine if a teacher should stay or be fired. Then again, it could be how my state (Louisiana) has implemented VAM. Our Governor Jindal, with a little too much support by our Legislature, passed laws that require a new teacher get 5 years in a row of ‘highly effective’ ratings to earn tenure (highly effective is defined as the top 10% of teachers in the state). A single instance of an ‘ineffective rating’ and you can scrap tenure. The chances of that happening are very, very rare (.001%, I believe)
Another false sense of security comes with the miraculous ability of VAM to factor in multiple variables that tug at all of our hearts—-at risk factors such as free/reduced lunch status, prior discipline, prior attendance, prior test performance, etc. Listening to the webinar on VAM by one of the LDE’s research scientist who was hired to develop VAM for LA, I even felt a little ease—–but then his two year stay at the department ended and he is no longer working on that project. So, LA teachers have no guarantee that VAM will be implemented as they were once told via this reassuring webinar. It is difficult to find a single person that can answer this question:
HOW EXACTLY DOES A STUDENT’S ‘GROWTH GOAL’ GET ‘ADJUSTED IF HE/SHE MET X NUMBER OF AT RISK FACTORS?
An even better question:
WILL THAT GET PUBLISHED?
When I asked the first question, I was told that it was not known until all of the students in the state take the state tests. Once all of the scores are in, an average performance for, say, 4th graders, can be determined. Then the magic happens and lil Johnny’s ‘growth target’ gets adjusted based on the number of at risk factors. I’m sure teachers would prefer to know how the magic happens.
Besides, in this feels-good era of ‘everyone goes to college’, how quickly can an administration publish proof that goals for some are different than goals for others and keep a straight face?
LikeLike
Today I’m angry, disgusted, demoralized,and frustrated. I am also firmly resolved to fight back against the tsunami of junk ideology that all good educators face these days.
I received my ‘growth score’ today from the New York State Education Department.
I know, I really shouldn’t care what my score is. I know 100% of my students tested at or above grade level in Math and English Language Arts. I know my class’ scores were near or at the very top of my district’s scores. I know my district is also at or nearly at the top of the region’s and states’ scores. I know I work my heart out and push my students to excel. My students always, ALWAYS succeed.
Yet according to the NYSED my growth score is so so. I’m rated effective with a growth score of 14 out of 20. Keep in mind, my student’s mean scale in math is 708.4 and ELA it is 678. I’m confident both scores are well above that state mean.
So why did I get a mediocre growth score?
The state’s explanation of it’s calculation should be a eye opener for all of us. Check out this junk math.
Click to access Teachers_Guide_to_Interpreting_Your_Growth_Score.pdf
Here it is in a nutshell..
They compare your students with similar students and measure how your students do to these similar students. You are then graded based on how much better your students did or how much ‘poorer’ your students did than these other students. They look for the gap between your students and the representative group of similar students.
Here the flaw…
If the representative sample of student all do well, your ratings will be negatively affected, because your growth is based on only how much better your students did than the group. In other words they look for a gap between your students and the group.
We all know that this year scores went up for everyone.. so as they rise, individual teachers get lower ratings, because the gap doesn’t increase. Sounds nuts doesn’t it? Goes against all the jargon about closing the gap.
It gets worse if you happen to have some high performing students in your class as well. Not much room for growth if you’re near the top, and your group is near the top. It’s a teacher’s advantage then to not take those high performing kids, It will hurt their growth scores.
My students did great, it’s a shame that NYS thinks they did so, so. Perhaps, if my students understood pineapples and hare races a little better, they could have correctly answered just 1 more question in that 6 hour marathon of testing correct, and all would be well.
We have a choice, we could start practicing saying, “welcome to Walmart”, for our next career or fight back. What say you?
LikeLike
Yes, I have been preaching this to my colleagues. Teachers are now enemies. The better the scores of students are for teachers that teach the same subject, the harder it is for any individual teacher to show growth. Also, some VAM systems make use of past data points and projects and calibrates a line of growth off of those historical scores. So not only am I at enmity with other teachers that teach my subject, I am at enmity with the teacher that teaches the course directly before mine.
So all that research for the last couple of decades, leading to the formation of Professional Learning Communities, now gets thrown out the back door.
There’s no way I’m helping ANYONE. It’s every man and woman for themselves – cut throat, corporate ideology now strewn in our schools.
LikeLike
rratto: You nailed it in your next-to-last paragraph–“if my students understood pineapples and hare races a little better…”–it isn’t simply the VAMs that are junk, it’s the very TESTS that all this is based upon
that are junk! As a special ed. teacher who administered these tests for years (and IL uses Pearson), I was able to look at the tests (because we read the math and science from scripts), and I can tell you that there have been NUMEROUS faulty questions and answers (more than one correct answer, NO correct answer) over numerous years. This holds true for all the prep garbage we buy, as well. Last but not least, people who score the written portions of tests (extended response on Reading Comprehension selections) are often people not competent to do so–such as ESL adults, who did not understand the nuances of the English language.
(As I have before, I must reference Todd Farley’s must-read 2009 book,”Making the Grades: My Misadventures in the Standardized Testing Industry,” which will explain in detail).
Farley also wrote a Huffington Post article this year about COMPUTERS scoring written essays (another must-read).
When you factor this together–along with all the score-juking that goes on (again, read the book!), WHAT “valid,
reliable” basis is there upon which to base a VAM?
It is all a house of cards–just like the wall that Diane predicts will
collapse. This bodes well for us–public schools and teachers–as this is the basis of our fight. Lawyers, ready yourselves!
LikeLike
If you apply to WalMart, don’t fill in your public school teaching experience. You won’t get the job. The Waltons are 1% Michelle Rhee/charter school fans!
LikeLike
All teachers should boycott Walmart….I refuse to shop there anymore.
LikeLike
Thank you for this conversation, which truly has to happen on a much larger playing field. Whenever the Chicago Teachers Union decides they have the best contract they can get at this time (and THEY are the ones who know best!!), as they have said, the fight for public education continues — everywhere. Their strike will have changed the conversation (yes!) and helped to level the playing field (bravo!) — but it remains for us to continue the struggle, together, everywhere.
I began this school year with a sense that this is the time teachers were going to begin to take back our profession. Now, with the CTU’s example and impact, I am confident of this. The conversation here is an example, and one we have to make bigger and more public. Thank you, Diane Ravitch and “science teacher” and all who are pushing this discussion. Truly, we need one another and we now have some newly-opened doors for our voices to be heard. Thank you, CTU!
LikeLike
Frankly the politicians and corporate reformers who are pushing VAM could care less about its validity or reliability. VAM is a convenient tool for ending tenure and career teachers, thus saving megabucks in state pension budgets. The irony is that if we were really in an “education crisis” due to the abundance of horrible teachers, we should be hesitant to fire even one great teacher over a bogus VAM ranking.
Teachers in Miami may start losing their jobs over VAM next summer (even though the state has still not released last year’s VAM rankings). As soon as the first teacher is fired because of their value added ranking, we will be able to file a lawsuit. VAM will never stand up in court. When we win the first lawsuit, they will have to end the VAM insanity. The way to end corporate ed reform is through the courts. It’s a shame that the millions of dollars spent on VAM algorithms in Race to the Top states will all go to waste. American Institutes for Research (a DC “nonprofit” selling the VAM algorithms across the US) will be laughing all the way to the bank.
LikeLike
If VAM is not valid, why would it be acceptable to use as a diagnostic?
LikeLike
I never said it should be used as a diagnostic.
It is not valid.
LikeLike
I was asking of the reader whose post you quoted from. That reader says that it would be ok if used as a diagnostic at the beginning of the third paragraph.
LikeLike
My state has been giving out growth data since 2009, but that data was only used as in a “diagnostic” arena. This meant that it told us, as an aggregate, whether our students “grew” or not. It meant little to us of course, but it included nice, colorful graphs.
The interesting part is to see how many teachers received all 3 ratings over the course of 3 years. Were they really good teachers, average teachers, or really bad teachers? We couldn’t really say, and we laughed about it.
No one is laughing now. That same inconsistent, unreliable data, which was pointed out by the OP as being invalid, is being used in association with high-stakes. This goes against the advice of several research bodies and many well-known, well-respected researchers (Ravitch, Darling-Hammond, etc…).
While it may make for yet another piece of interesting diagnostic data, it has no place determining whether a teacher is effective or not, on any scale.
LikeLike
It’s not acceptable in my eyes. Start with caca end with mierda. The whole process has so many errors involved that it is completely invalid. See retiredbutmissthekids response above about these tests.
I was forced to give the SAT9 one year to a freshman class. I read/analyzed the test even though I wasn’t supposed to. Sorry but I refuse to give a test without reading through it. There were so many errors in the questions in every section (minimum 15% error rate maximum over 50% error rate) that it was a complete joke. No wonder they don’t want the test givers to read them. They’d be called out for the fraud that they are.
A couple of times a month I tutor after school watching, helping students do “credit recovery” on line. Boy is that a joke, questions worded improperly, not accepting 1 1/2 for 1.5, lots of BS trivial questions etc. . . . It’s a sad day that we have come to this point.
LikeLike
Reblogged this on CENTURY21SCHOOLS.
LikeLike