Here is the story of the Houston Seven, the teachers suing to invalidate the evaluations based on student tests scores.
How nutty is this?
“Andrew Dewey is an award-winning history teacher at Carnegie Vanguard High School in Houston. In 2011-12, he earned the top merit pay award that his school district gives out and had “most effective” teacher status through a controversial evaluation system that uses student standardized test scores. The next year, after teaching similar students in the same way, he went from being one of the district’s highest-performing teachers to one that made “no detectable difference” for his students.
“Dewey is one of seven high-achieving teachers who, along with the Houston Federation of Teachers, filed a lawsuit in federal court in Texas late Wednesday alleging that the Houston Independent School District uses a badly flawed method of evaluating teacher effectiveness, known as the “Educational Value-Added Assessment System.” The teachers argue that the EVAAS is inaccurate and unfair but that it still plays a large role in determining how much teachers are paid and whether they can keep their jobs.
The method, generically known as “value added measures,” or VAM, is increasingly in use around the country — with the support of the Obama administration — after Michelle Rhee pioneered the method when she ran D.C. public schools several years ago. The result of this lawsuit could affect evaluation systems well beyond Texas.”
Just think: if the Houston teachers win, and the evidence is on their side–they take down the central theory of Race to the Top and Rhee, as well as laws in dozens of states that will face similar lawsuits.
“Randi Weingarten, the president of the American Federation of Teachers, said that the obsession with standardized testing that has driven education policymakers to make standardized test scores the key metric of accountability for students, educators and schools, is bastardizing public education.
“This country has spent billions on accountability, not on the improvement of teaching and learning at the classroom level, and value-added models are the leading edge of this misguided effort,” she said.”
Start with Race to the Top. $5 billion wasted.
Diane Ravitch's blog 
This is very good news. Of course, it is important to put forward the alternative, for this is the first thing that deformers ask: What’s your alternative?
The answer: collaborative, integrated peer evaluation as part of a continuous improvement procedure that is teacher-led and passed in something like Japanese-style Lesson Study
Teachers in districts that were awarded RttT funds should be suing their states and Duncan as well.
you are wrong….they should be suing their education union leaders,.,..they were the ones who signed out with the politicians in the state legislature…the commissioner of education, the teacher unions and then the state general assembly were all part of RacettTop.
Blame the clueless teachers who did not do due diligence and read the 150 page RacettTop document. I read it when it first came out. I saw what was in it. I saw the evaluations, the high stakes testing, the tying of test scores to a teacher’s eval, the common core curriculum….etc etc…All that was in Race ttTop but teachers afre not proactive and they deserve what they got…HAd they read it, they would have protested to their union leaders who would not have caved in like they did in my state.
YOu have to read documents before you sign them. These people were dumb not to read what was in RacettTop…They get what they deserve.
Would you sign insurance papers or bank documents without reading through them first? Of course not.
Well, the teachers should have read what their part in this documentation was and then say NO before it got to the Duncan.
Duncan provided the money—the state with its teacher education group made the decision to sign up for the money….I dislike Duncan and his policies big time but it is like the tree in the Garden of eden…People got greedy and they wanted the money…the millions….too bad now and too late…you reap what you sow..
The state got the money with the belssings of the players involved and now your union and your board of education in your state with its commissioner in addition to your legislature and governor sold you down the river.They are making you pay for this money with absurd high stakes testing and CC standards and unrealistic evaluations..and the money doesn’t even go to the school or for supplies or for computers–it goes to teacher training porgrams..the biggest crock in the history of the take over of education by privatization people….
To blame the teachers for RaTTT is way off track. No teachers were involved with the various MOUs signed by two people for each state, the governor and the top administrator of the state education agency. You’re dead wrong in blaming the teachers.
However, you’d be dead right for castigating the teachers as GAGAers for acquiescing to and implementing these educational malpractices with nary a peep against them nor refusing to implement them.
I’m certainly upset with the union leaders (talking national ones especially) who signed on to this, but I as a teacher — though contemporaneously well aware of how bad Race to the Top was — and really my entire school district had zero influence in the state’s decision to accept RttT and its funding.
There’s a lot of blame the victim in our current educational culture.
You are wrong. You make a lot of assumptions. I do not belong to a union and never have. Teachers in several states have no union protections. Stop scapegoating teachers.
I for one do not belong to the union as I live in a “right to work” state and a Bush one as well. (Florida). We were never asked if we approved RTTT. It was voted by our LEGISLATURE (who want to do everything they can to disband public education) and shoved down our throats whether we liked it or not. I have not talked to one teacher who approves. WHY IN THE WORLD IS EVERYTHING WRONG IN THIS COUNTRY always placed at the foot of teachers???? Is it because everyone has gone to school so everyone knows how to teach? I sit here night after night feeling like I am living in the Twilight Zone reading some of the comments from people who have no idea what it is like in trenches. We should have told them no??? OMG we have been screaming for a year………..but notice this is one of the few places that listens. Thank God Diane can’t be bought.
In my state the teachers were the ones that supported it. In your state you did not have that luxury. You have the right to sue your legislature. But you don’t have the right to make a blanket statement like you said:
EVERYTHING WRONG IN THIS COUNTRY always placed at the foot of teachers??
Teachers , my friend are not lily white, smart and savvy as you seem to think they are…And for you to say everything is based at the foot of teachers is a grandiose exaggeration but in some states teachers are their own worst enemy.- mine being one of them…I’ve been in the trenches, as you call it, friend, for over 30 years….I have been on the front lines and yet the people I worked with were not all on the same page…They would not fight against CC or RacettTop. They will not fight against privatization and pension changes. They believe wholeheartedly in their union leadership without questioning it…Look at the pension fiasco in my state…Teachers could have changed it and voted the draconian RI Pension law of 2011 down, but they didn’t…
You live in a tough state. I don’t envy you your position. Jeb Bush is a piece of work and he has put one of his Chiefs for Change in my state…Debbie Gist is RI’s Commissioner of Education. The beliefs of people here are not the same as where you are .The percentage of sheeple is high here!
Luckily we don’t have a “Jeb Bush moron” running our state but who knows what we will get in the future.
There are no unions in Texas and Texas did not opt into Race to the Top.
Call me cynical, but I’d be more impressed with Mr. Dewey if he had protested the evaluation system when he was rated “most effective” and before he took the merit pay. If the system is flawed (and I agree it is), then it’s no more valid in picking the “high achieving” teachers than it is in picking the “no detectable difference” teachers. By suing now it only looks like sour grapes.
I’m not sure he would have had the standing to sue if he had done it when he had a high rating, because then he would not have “suffered harm,” which is what courts require in order to sue in the first place.
FLERP! or Labor Lawyer or any other lawyer on board,
Is what ToW says is true about the “standing to sue”?
Thanks!
Yes, standing is legal requirement. The plaintiff must experience a legally recognizable harm or damages. Without damages, the case will not proceed beyond summary judgment.
Yay! I remembered my Constitutional Law classes!
Dewey and the local union have been fighting EVAAS on many fronts since it was brought into HISD, and it didn’t have anything to do with just his score at any particular point in time. Remember there are six others suing, along with the local union. The timing was more an issue of having a solid group of plaintiff’s–who, yes, have suffered harm–and the legal arguments necessary for an effective suit. Your notion of “sour grapes” is way off base. These teachers have consistently spoken out against VAM and the EVAAS system.
You make an interesting point, Dienne, but I’m giving the benefit of a doubt that in being “most effective” one year and “no detectable difference” the next, it helps to show the absurdity of the system.
It most certainly does show its absurdity.
You really do not understand VAM if you believe this. It is very possible for a teacher to be most effective one year and NDD the next. Do you teach the same students every year? Is every group the same?
@JM Bartels, I confess to understanding as little about VAM as any of the so-called experts do. To your point of having different groups, I was the head coach of a varsity soccer team for many years. Two of my teams won state championships. One of my teams went 4-10-4. While I would contend my coaching was equally effective (certainly insignificant differences of effort level and time spent developing the boys — on and off the field), if you measured my effectiveness simply by record, you’d say there was no comparison between my effectiveness in the championship years and my losing season. But judging a coach’s effectiveness solely by the record is an overly simplistic analysis that often misinterprets what is really happening. My view of VAM is that it is of even less use in judging a teacher than a record is for judging a coach.
While I certainly agree that a teacher’s VAM scores can and likely will vary widely for numerous reasons (many beyond the teacher’s control), I disagree with your inference that the teacher’s actual value is that different from year to year…simply because the VAM says so. My experience is that a good teacher this year will generally be the same good teacher next year. I have never experienced a teacher who was great one year and poor the next (or vice versa).
In the end, JM, our (respectful) disagreement is over how effective VAM is at measuring the value and effectiveness of a teacher.
Quoting
“This country has spent billions on accountability, not on the improvement of teaching and learning at the classroom level, and value-added models are the leading edge of this misguided effort,” she said.”
District lawsuits cost money and school board confidence in strategy and direction. Enough doubt about legality expressed by school boards should shelve the process.
This is another effort that can not be judged to be first things first for any district in reform. School boards should want to get back to an undisputed process that gathers support from the communities at large.
This whole direction on the teacher verse the student is sad if you are a reformer that knows it is very important to do first things first directly with the children.
I tried to do the same here in CT, but no attorneys had the necessary courage to help us out. My hat’s off to Houston teachers!
Let me work off the quote from Randi Weingarten that “standardized test scores” are “the key metric of accountability for students, educators and schools” and that this “is bastardizing public education.”
The 52nd anniversary of a public statement that sums up 2014 in public education:
[start quote]
All methods of evaluating people have their defects—and grave defects they are. But let us not therefore allow one particular method to play the usurper. Let us not seek to replace informed judgment, with all its frailty, by some inexpensive statistical substitute. Let us keep open many diverse and non-competing channels towards recognition. For high ability is where we find it. It is individual and must be recognized for what it is, not rejected out of hand simply because it does no happen to conform to criteria established by statistical technicians. In seeking high ability, let us shun overdependence on tests that are blind to dedication and creativity, and biased against depth and subtlety. For that way lies testolatry.
[end quote]
From the very last paragraph in Banesh Hoffman, THE TYRANNY OF TESTING (2003 paperback edition of 1964 edition of 1962 original).
Can’t say we weren’t warned way ahead of time…
😎
Using student test scores to evaluate teachers is bad policy — too unreliable with many false positives and many false negatives as well as many serious adverse side effects on the quality of education generally. But, a lawsuit is probably at best a long shot. Courts are usually reluctant to second-guess an employer’s decision to use evaluation system A rather than evaluation system B. Recognize how counterproductive So long as the employer applies the system correctly and consistently (and so long as there’s no inherent discrimination against a “protected” class, the courts will usually not get involved. In other words, absent a specific statutory or contractual provision, it’s unlikely that a court will strike down a fairly-applied system just because it’s a dumb system. Those of us who recognize how counterproductive high-stakes testing is will probably,have to fight and win our battle at the polls or in the superintendent’s office, not the courtroom.
As I noted in your Washington Post comment, read the lawsuit for yourself at http://tx.aft.org/files/houstonevaas_lawsuitfiled.pdf It’s based on violation of due process, since teachers can’t challenge the EVAAS scores and other factors.
Thanks for the link. The complaint is well-written. However, other than the allegation that HISD is slanting the instructional practice ratings for the low test score teachers (a due process violation), I think it’s a long shot that a court will want to. Second guess the school system’s decision to use the VAM to evaluate the teachers. The standard for a substantive due process review should be something along the lines of reasonableness with a lot of deference given to the govt actor. Otherwise, we turn the DP clause into something that let’s the courts rather than the legislature or the executive decide how to address society’s problems. It was substantive due process analysis that let the SCt overturn the first round of New Deal legislation and that, if not rejected for political reasons the next year, wold have prevented FDR from implementing the New Deal. Our answer to high-stakes testing is at the polls or in the superintendent’s office.
I am not a labor lawyer. My practice was limited to family court and bankruptcy matters. What about the argument that a teacher’s employment is a property interest and termination based on VAM is an arbitrary and capricious deprivation of that property interest (given the statistical flaws in the methodology) Have you had chance to read the complaint? I would love to know what you think after you see the complaint. Do we know how many actions are currently pending? Any resolved?
What strikes me is how any teacher who was so excellent can be deemed useless a year later.
I experienced this long before VAM. I was the EDUCATOR OF EXCELLENCE CHOSEN BY THE NY STATE ENGLISH COUNCIL, based on the observations from the Pew funded AUTHENTIC National Standards research, which matched my practice to every single indicator for learning in the Harvard thesis which was the research. IN fact, of 20,000 teachers studied, I was one of six that matched all the criteria in a unique way; the LRDC at the Univ of Pittsburgh, that are national staff developers on the ph’d level, sent my work around the country to show teachers what BEST PRACTICE looked like.
… BUT, one, lone superintendent in NYC decided I needed to be sent packing so test prep and other mandates could replace real learning curricula, and she made up the subjective criteria and charged me with incompetence. Now, 15 years later, this method for slandering teachers is incorporated into the PROCESS. Look at LAUSD
http://www.perdaily.com/2014/03/lausd-and-utla-collude-to-end-collective-bargaining-and-civil-rights-for-teachers-part-2.html
Unfortunately too often job security depends on how effective a kiss-up you are. I know that there are socially acceptable ways of phrasing this idea. Being a team player comes to mind, but really it comes down to not irritating the powers that be to a point when you are more than annoyance. You are very easy to replace with someone who will follow the party line. The value of consensus has been overrated. It has morphed into forcing acquiescence.
Thank you “2old2teach”. All superstar “2young2learn” people who grow up with their open minded, make the same mistake with assumption that their BOSSES are as smart, and as open minded as their parents.
In short, please do not work for any boss who has less quality, less capability, and less class than oneself. We are intelligent enough to find our way out before our frustration will kill our health.
It is so sad to acknowledge that the education system in the richest country in the world can be bought and destroyed by a handful of businessmen with a complete disregard for the welfare of public democracy. Sigh. back2basic
It happened to me. Two years ago I was raved about. And now, not so much. Thanks, Danielson.
Weingarten is a hypocrite to now advocate about testing. She knew from RacettTop implementation that high stakes testing and evaluations attached to that testing was in the document…If the state wanted the money from the feds, they had to adhere to the rules of the game.
Weingarten also in the past has defended CC and Common Core includes high stakes testing. She is a flip flopper and has alligned herself mroe with the politics of things rather than the classroom environemnt and school education part of things.
These leadership peopple have not been teaching in a classroom for years and thus “forget” what it is that teachers go through on a daily basis…And thr teacher of 2014 has it differently than when Weingarten and many teachers were around in the 80’s and 90’s and even in the early 2000’s…We had evaluations. We had standards. We did not have the corporate take over- we did not have ALEC. We did have sympathetic leaders back then….not today. Union lkeadership does not have the same mindset as rank and file. Look at RI as a perfect example. The treaurer and the Union leadership made a 14 month long backroom deal on the pension and leadership agreed to the final settlement agreement deal. It was a rigged agreement and the teacher union leadership threw their members, who pay high dues to these union leaders as their salaries, under the bus…and took away representation of those union members who voted no…they would not be able to litigate or go to court..And where was Randi in all of this? She’s nowhere to be found…too busy going around the ocuntry making politcal pacts securing her own future.
And yet it is the RI pension crisis in the news. The whole country is looking to see what the courts will rule since it is thankfully going to court…the settlement agreement got voted down- but not by the teachers but by another group in the coalition. Teachers here get intimidated and bullied by leadership or simply thnk their husbands will take care of them.
Attitude like that is prevalent…helops the leadership but not those who need their pay, pension and cola to survive….And the way the SA was put together was not ethical and out in the open but secretive and not transparent…with more lawyers, retirement board members and coalition leadership in those 14 month long negotiations that the ones that made money were the negotiaters and lawyers….
“Teachers here get intimidated and bullied by leadership or simply think their husbands will take care of them.”
That’s the kind of sexist attitude that was prevalent before the Women’s Rights Movement and, unfortunately, it still prevails today in too many places, despite the fact that a lot of teachers are the only bread winners in their families, regardless of gender.
I’m sorry if your pension is at risk, but at least you have one. Many of us don’t have pensions. I would think you could spend better use of your retirement advocating for teacher pensions instead of trashing teachers.
I do advocate for teacher pensions in my state…Not trashing teachers…just letting you know not everyone got into the teaching purpose as you and I did…you lump everyone in one group…can’t do that anymore….too many without the proper training and skills and are in classrooms unprepared…Don’t be naive….there is good and bad in all professions…and the privatization people are taking full advantage of what’s going on….
Like teachers don’t already know there are some rotten apples amongst us? I don’t need to be told that. I’ve spent much time in my career trying to educate bad teachers, and, when they didn’t improve, a lot of time trying to get rid of them as well. Fortunately, they have been the exception, not the rule, and I value my many colleagues who are consummate professionals. You make sweeping accusations against entire groups of teachers and then claim that others are doing that. Get off your high horse and look in the mirror.
Randi confuses me often. I low she is a political person… has to be to reach her position, but I often wonder about some of the things she says as opposed to some of the things she actually does. She is not a bad guy, and she has so much potential, if she stood her ground for us!
Making the Grades: My Misadventures in the Standardized Testing Industry by Todd Farley, ISBN: 978-0-9817091-5-4
(pg. 242)
“If I had to take any standardized test today that was important to my future and would be assessed by the scoring processes I have long been part of, I promise you I would protest; I would fight; I would sue; I would go on a hunger strike or march on Washington. I might even punch someone in the nose, but I would never allow that massive and ridiculous business to have any say in my future without battling it to the bitter, bitter end. Do what you want, America, but at least you have been warned.”
LLC1923: TAGO!
😎
Again, I’m confused.
To be clear, I’m opposed to using test scores as the basis for teacher evaluations. Value-added models are inherently unreliable and problematic.
Having said that, lets’s consider the case of Andrew Dewey who is cited here as one of the Houston VAM litigants.
Both here and at The Answer Sheet (Valerie Strauss’ blog at the Washington Post), Dewey is cited as a “top merit pay award” winner. But I see not a single question raised as to whether or not Dewey took his merit award when Houston evaluations ranked him highly. Did he? And if he took merit awards earlier, has he now offered to give them back since they may well be based on bogus calculations? If he took merit awards earlier, then why is he complaining now?
Dewey teacher Advanced Placement History at the small magnet Carnegie Vanguard school in Houston. The school is for “gifted and talented” students. Not just anyone can go to Carnegie; students have to be “identified” through the Houston “gifted and talented” screening matrix, or they have to be tested (and pass) to gain acceptance. Carnegie Vanguard prominently touts its ranking among the “best” and most “elite” schools in the country. In a very real sense, Carnegie Vanguard is a poster child for what’s wrong with American pubic education.
Compare to the rest of Houston and to Texas, Carnegie has a very small “at risk” population. Last year, “1300 students applied for admission for a freshmen class size of 188.” Carnegie flaunts the number of National Merit Scholars and AP scholars it has as though this means the school is doing really good things.
As I’ve noted previously, the PSAT and SAT –– produced by the College Board –– measure nothing much more than family income. The AP program, also College Board produced, may be beloved by many, but it’s mostly a myth. Research consistently finds that AP just doesn’t measure up to the claims made about it. For example, a 2006 statistical analylis found that Advanced Placement did not reach “the threshold level of significance” as a proxy “for curriculum intensity.” And a 2010 book (AP: A Critical Examination) concludes among other things that”despite the immense popularity of the program, the research evidence on its value is minimal.”
The College Board and the ACT were both major players in the development of the Common Core. Both have “aligned” their products with it. But their products are more than a little educationally suspect.
I’m not a VAM fan. Nor a fan of the Common Core. Or of Randi Weingarten (who helped to create the current mess). And I’m not particularly fond of hypocrites. And there sure are a lot of those in public education.
No doubt the AP program could be better, but it seems to do a reasonably good job preparing students, I wish that more of my staid ends had the opportunity to take AP classes while in high school. The exam format also allows students to demonstrate competence in a subject when they have not taken conventional classes. My middle son, for example, took three AP exams on material he studied for independently of any class, two exams that were related to the class but not the focus of the class, and one based on a virtual class run by K-12.
I know you like AP courses TE, and perhaps they worked well for your son.
AP may indeed work well for some students, especially those who are already “college-bound to begin with” (Klopfenstein and Thomas, 2010). As Geiser (2007) notes, “systematic differences in student motivation, academic preparation, family background and high-school quality account for much of the observed difference in college outcomes between AP and non-AP students.” Even the College Board concedes that “interest and motivation” are keys to “success in any course”. Klopfenstein and Thomas (2010) find that when these demographic characteristics are controlled for, the claims made for AP disappear.
What the College Board doesn’t like to admit is that it (along with the ACT) sells “hundreds of thousands of student profiles to schools; they also offer software and consulting services that can be used to set crude wealth and test-score cutoffs, to target or eliminate students before they apply…That students are rejected on the basis of income is one of the most closely held secrets in admissions.”
The College Board and the ACT are inextricably tied to the Common Core, and to the assessments that will ultimately follow. And of course to their products, which are “aligned” with Common Core…the ACT, PSAT, SAT, and AP.
That’s why I questioned the posturing about some of the characters in the Houston VAM lawsuit. They’ve helped to perpetrate nonsense, and even received “merit” bonuses for doing so. Why weren’t they questioning the system before?
Public education is in serious trouble. It has some very poor “leaders” (the heads of the main teachers unions “professional organizations). And far too many educators have bought into educational myths.
So if it works well for some students, it seems reasonable to offer those students the opportunity to take those classes. Unfortunately the majority of high schools in my state are too small (under 250 students) to offer AP classes.
Does he need to give back the high pay received previously when he…apparently…liked the system? Is it like coaching…”when team looses we have a bad coach”.? I have students who literally on test day put their heads on the desk …looked up…marked any answer. My fault? I did all I could before testing, with parents, in test session without in validating test for everyone else, but the student could have cared less. Judge me on that? Make my pay or incentive pay based on that? Sooooooon very soon…there will be no one choosing to teach!!!
EVAAS is one of the most robust and fair of all of the VAM out there. I don’t understand the resistance of teachers to this, given that it is measuring the teachers influence on student growth, as opposed to student achievement. Student achievement can very well be predicted by looking at the zip code of where the student lives. Now THAT is unfair. However, EVAAS takes into account many data points over time, and is a very comprehensive measure of a teacher’s influence on a GROUP of students. Statistically, it takes into account a large data set that accounts for individual outliers. It includes a measure of error, or confidence interval that allows for random fluctuations. Based on the comments I have read here from teachers, it appears that many just do not understand the measure and would prefer to rely on very suspect evaluations by principals or peers which have been shown to be VERY unreliable. I am disappointed in my teaching friends who criticize something that they have little understanding of.
JM Bartels, Any teacher evaluation system based on student test scores is Junk Science.
JM, I am interested in your background for the context of your support of VAM. Are you a teacher? If so, have your VAM scores matched your own views of your effectiveness?
I have very little faith in VAM, and I think this case is an anecdotal example of its weakness (I have heard of many others). Even if I thought VAM did fairly measure a teacher’s “effectiveness” in teaching the tested material (which I don’t — the PARCC tests certainly don’t), I would question its worth in judging how valuable a teacher is. So much of being a truly effective teacher can’t be measured. And the quest to reduce everything to a number has created unintended (if predictable) consequences that demonstrate there aren’t simple solutions to complex problems in education.