Audrey Amrein-Beardsley has been consulting with the seven Houston teachers who filed a lawsuit in federal court against the use of value-added metrics in their evaluations.
She has conducted extensive VAM research in Houston and concluded it was arbitrary and inaccurate. “Houston, the 7th largest urban district in the country, is widely recognized for its (inappropriate) using of the EVAAS for more consequential decision-making purposes (e.g., teacher merit pay and in the case of this article, teacher termination) more than anywhere else in the nation.”
She believes that this is the lawsuit that has the potential to bring down VAM as a valid way of measuring teacher quality.
If VAM goes down, as it should, it would be yet one more piece of evidence that Race to the Top is a $5 billion flop, as if any more evidence were needed.
Of course, even a court victory against inappropriate teacher evaluation would not deter our Secretary of Education from claiming victory. If he were on the basketball court, he would claim victory if his team were beaten 152-18; we would never hear the end of those heroic, astonishing, incredible 18 points.
Diane Ravitch's blog 
Here is something I wrote about the FL VAM sham:
——————————————
The Golden Rule of Accountabilty:
VAM unto others as you would have others VAM unto you
To: School, District and State Educational Administrators
From: Your Passionate, Professional and Committed Teachers
Re: VAM (Value Added Models, or Very Arbitrary Measures)
Dear Administrators and Pedagogical Theoreticians:
Since you believe teachers should be held accountable for results and have a proportion of their paycheck tied to test results, then you must also believe that you too should have part of your paycheck determined by test results. If you disagree with this then tell us why; give us a rationale for your hypocrisy.
Teachers complain that VAM statistics ignore some basic variables (ex. socioeconomic levels) and that they will be blamed for poor performance (aka: low test scores) for factors that are beyond the control and scope of their classrooms and influence. They will even point out all the variables that lie within the affective domain (desire, perseverance, work-ethic, love to learn, will to self-actualize) are primarily developed within the domain of the family (after all, family support-input is the primary predictor of success in school and post-secondary pursuits). Yet, under VAM teachers will get the blame for low performance, even though the equations are supposed to consider primary factors (but ignore socioeconomics and family). So even when we do our best some kids will not learn in spite of our efforts, though our paychecks will suffer.
So, then you too dear administrators must get blamed for events beyond your control, because if “it takes a village to raise a child”, then all in the village should suffer the negative reinforcement (lower paycheck) when the child fails.
We, teachers, may decide to blame you for your possibly deficient leadership or ineffective mentoring, or buying junky curriculum, or…? If, in the metaphor of the learning-village, part of our effectiveness as teachers is tied to your effectiveness as leaders and guides, then you too should be held accountable!
Are VAM (Value-Added Model, or Very Arbitrary Measure) truly unbiased and take into account as many covariates, or confounding variables (and factor these out), so that the “signal to noise ratio” is significant. One analysis showed that only 12% of test scores was affected by teacher input, the other 88% to non-teacher factors.
Is the value of helping out a low income, inner-city, child go from a 1 to a 2 in the FCAT, the same as that in a rich suburb of going from a 3 to a 4? Intuition would say no; that it might take more work (and therefore more value?) to help out the student who grew up in a less-fortunate environment. Will VAM statistics factor this into account, and if so, how, and if not then they are seriously flawed. Should VAMs be handicapped, so that gains made in the lower 1/3 of the student population are weighted more than the upper 1/3, because it takes more teacher-effort to raise the lower 1/3? If little or no gains are made by the upper 1/3 is it because they have poor teachers, or that it is more difficult to create gains with groups already in the 80-90 percentiles?
If we “VAM unto others as we would have others VAM unto us”, then what kind of rubrics do we use with administration, district and state education leaders, and the legislators that make the mandates ?
District administrators may feel this is unfair. Some of you might even admit that you choose educational leadership not because it was the culmination of your passion for teaching, but because you would get a bigger paycheck. Some of you could even get hit with the epiphany: “if after only teaching for 3-5 years, and that with no documentation of excellence, I am suffering no accountability for VAM, yet the teachers are, I feel somewhat uneasy; something is not just”.
You may complain that variables beyond your control should not be the reason for you lower paycheck under a leader-VAM statistic, but sorry no double standard. District leaders may grumble that FL DOE leaders, or FL legislators, made choices that negatively affect their performance at the district level. Well, so, should you not be held accountable anyway?
FL DOE leaders may blame the federal government for poor funding, bad curriculum mandates, unsound/invalid pedagogic assessment models, and whine that getting smaller paychecks is unfair.
After all, should we not all just blame the President? No, of course not, individuals should and must be held accountable for producing excellent products; the “buck must stop somewhere”, agreed. Yet, why does it stop at the teachers? Why are we the only ones who will have a part of our paycheck tied to a VAM statistic? Why not leaders too? Why not parents too? Hey, why not penalize the future salaries of students who deliberately choose not to learn with a student-VAM?
But no, we, the teachers, will take all the blame for failing students (not even the parents get blamed) without any accountability (tied to salary) for our leaders? We are flattered that you leaders believe we have that kind of power in the classroom; that we can lead the horse to water and make it drink; that we can plan and cook the meal and make someone eat it too.
How we wish, as teachers, the assimilation of knowledge was so easy; that all our input equated to student output, but we all know this is false.
Does not Maslow’s hierarchy of needs teach us that no student will self-actualize and desire to learn for the delayed future reward of a good job, or the “love of learning”, whose underlying primary needs of love, support and home-life are not robustly provided. Students today, in general, come into the classroom with so much “baggage” (ex. uncertainty of family support after 2 divorces, abusive authorities, excessive premarital sex, an Internet that exposes them to less-than-desirable behaviors, having some of the adults they used to look up to disappoint them, being bombarded with inane, vain and useless internet entertainment, etc.).
So, teachers are expected to produce a superior product in spite of the defects in the raw materials that enter the classroom? As a chemistry teacher I believe I can help the student (ore) refine itself, and will do so with all my passion, but I cannot do alchemy; I cannot make efficacious teaching and learning happen when the ore is unwilling to be refined.
Do we even teach and test on the affective domain of hard work, perseverance, diligence, honesty, self-sacrifice, altruism, grace, mercy and self-control? Yet, are not there variables as important, if not more, to being successful in the real world, are what most employers are looking for (not just cognitive potential). Yet, our VAMs never consider these variables, and therefore are limited in their predictive power!
Did our test-metric-engineers forget about the maxim: not all that counts matters, not all that matters can be counted, not everything counted has value, and not everything of value may be counted?
After all, what do test scores prove, if not nothing more than having good test-taking skills (ex. cramming to fill short-term memory and organized mental-schema that help one store and access data). Research shows time and again that there is very little transfer of knowledge, or skills, across content areas in high school students because their underlying knowledge base and mental-schema are still in the developmental stages and have not had enough time and experience to make the deep and profound connections, that occur with more maturation and study.
We believe students should analyze and solve geometry proofs and logic-tests, hoping that it will transfer to their language arts classes, whereby they will be more able to deconstruct texts and analyze authors’ intent and purpose. Yet, research paints a picture that is more correlational than causational; students that can think, analyze and produce do so in most classes, regardless of the teaching strategy or assessments used. Which again, proves the goal of a good liberal-arts pedagogy of “teaching students how to think, not what to think”. “How to think curriculum” is more messy and sticky; it requires long-term studies and research, interdisciplinary explorations and teacher collaborations; it is not so neat and easy as teaching a “what to think” class with an EOC. Yet, how-to-think skills are used more in the real world, than the short-term (shallow measure) memory skills of EOC tests and the curriculums determined by them. After all, if a test goes into VAM, and VAM affects paycheck, then teacher will more likely “teach to the test”, and the overall quality of education will suffer…..duh!
In the end of it all, the Big-Picture, I always point out to my students the “Graph of Life’s Predictions”, where the Predictors go on the X-axis (ex. GPA, AP classes taken, AP exams passed, FCAT scores, etc.) and Criterions go on the Y-axis (ex. future socio-economic class, contribution to the GDP, “being a prepared and responsible citizen”, “self-value/actualization/fulfillment”, etc.). Then, I point out that there is little, to none, long-term research and data to show that there is a positive correlation between predictors and criterion. We assume students with higher GPA will make a greater contribution to the nation’s GDP, but where is the convincing data. What about the outliers (if the real pattern is even known), like the Una-Bomber, who probably scored high in school, but was very-low in life?
In the Big-Picture, 90% of my chemistry students will forget 90% of the content on an EOC within 9 months, unless they restudy it because they take college chemistry. That can be said of most classes; unless the content is repeated and restudied (and then “permanently” assimilated), the learning was just temporary (and any short-term measure of it, ex. EOC: end-of-course exams) and has little to no effect and/or value in one’s adult life and career choice. So, why do we make such a big deal of EOC in high school, when they don’t even do this in college? If our tests have little power/significance in preparing, or predicting, success in the adult-life, then why are they such an important term in the VAM metric? Previous generations never suffered through all this testing, and teachers never scrutinized by VAMs, and yet the high school graduates of the 50s-80s were well prepared for college. So, what happened to the mantra: “if it ain’t broke, don’t fix it”?
So…..
Dear Superintendent, if we are going to get smaller paychecks because of an unjust VAM statistic, then why does it not work the other way? Why not divide up the money won for the Broad Award (or all other awards for education) and share the winnings with your teachers? Though, we might feel guilty because if we do get higher wages because of VAM incentives, then should we not share some of that with our students. For they, after all, were the ones performing on the tests. Just where does “the buck stop” or the reward end? Should some merit-pay go back to the families (their love, support, concern and accountability) that are the number one predictor of student success?
Mr. Governor, should not a proportion of your salary also be tied to student test scores? Is not the principle “a servant is not greater than their master” be applied to you? If we fail in the classroom, then your leadership must have something to do with it? Of course, the families never get any blame?
Mr. Scott if the FL GDP does not rise during your tenure, should not your salary be impacted? You may complain that federal policy has tied your hands. Well, do you now empathize with how teachers’ input is limited? Teachers strive for excellence, in spite of many of the socio-cultural variables that inhibit learning (ex. multiple divorces, inane and excessive entertainment medias, etc.). So, please treat us with knowledge and respect we deserve, and therefore drop VAM policies. If you disagree, then please have some statistician write a FL Governor VAM metric (and load it with variables you have no control over, ex. Senate bills passed) and then you might be more able to empathize with teachers.
Dear FL State Legislators, you approved VAM, so then apply it to yourselves. 50% of your pay should be based on the number of bills your author, and that PASS the legislature. We don’t care how hard you work, how many joules of energy you expend, or the product/profit you produce. No, we only care if your bills get passed (metaphor for test scores). Oh, but you will complain that the rate of bills passed is due to variables beyond your control. So, deal with it; suck it up and be treated in the same way you want to treat your teachers.
So, to all leaders who are unwilling to have a VAM equations applied to their own salary, STOP the hypocrisy and join us in a more justice and equitable society. Be willing to walk in our shoes, before you arrogantly and pretentiously tell us how they should be worn.
Mr. Rick Lapworth
Science Teacher, Felix Varela High School
15255 SW 96 St, Miami, FL 33196
Ed.S., NBCT 1999-2019
305-752-7900, x 3259, rm 259
Fax: 305-386-8987
Don’t forget courageous, Diane. Our Secretary of Education is always quick to note the courage he and his fellow reformers share.
Hope Texas fares better than Florida. We lost our second VAM realted lawsuit today as it was dismissed by a federal judge who acknowledged that it is unfair but not unconstitutional.
Ah, the state of the judiciary these days.
We don’t care about fairness, equity and/or justice!
No, the only thing that matters is if it is legal or not-of course by our own interpretation of the law in which we squeeze the meaning of words so hard until they appear to mean what we want them to mean and then by force of our “judgement” they legally mean what we want them to mean.
We judges, of course, being above the rest of you non-judges by reason of our own proclamations.
Duane, For the record, it was a federal court , not a state court. The reasoning behind the ruling, as Diane reported, leads one to question the sanity of the federal court.