Last week, the Houston Independent School Board deadlocked in a 3-3 tie vote on whether to renew its contract with the vendor supplying the teacher evaluation program.
Audrey Amrein-Beardsley explains their decision here.
At least three board members realized that five years of this program had not moved the needle by an inch. If performance matters, then EVAAS was a failure.
Beardsley is one of the nation’s leading researchers in the study of teacher evaluation.
She writes:
Seven teachers in the Houston Independent School District (HISD), with the support of the Houston Federation of Teachers (HFT), are taking HISD to federal court over how their value-added scores, derived via the Education Value-Added Assessment System (EVAAS), are being used, and allegedly abused, while this district that has tied more high-stakes consequences to value-added output than any other district/state in the nation. The case, Houston Federation of Teachers, et al. v. Houston ISD, is ongoing.
But just announced is that the HISD school board, in a 3:3 split vote late last Thursday night, elected to no longer pay an annual $680K to SAS Institute Inc. to calculate the district’s EVAAS value-added estimates. As per an HFT press release (below), HISD “will not be renewing the district’s seriously flawed teacher evaluation system, [which is] good news for students, teachers and the community, [although] the school board and incoming superintendent must work with educators and others to choose a more effective system.”
Open the link, read the full article, and read her links. This is excellent news.
The bad part of her post is the news that the federal government is still giving out grants that require districts to continue using this flawed methodology, despite the fact that it hasn’t worked anywhere.
Apparently, HISD was holding onto the EVAAS, despite the research surrounding the EVAAS in general and in Houston, in that they have received (and are still set to receive) over $4 million in federal grant funds that has required them to have value-added estimates as a component of their evaluation and accountability system(s).
So Houston will have to find a new vendor of a failed methodology.
Diane Ravitch's blog 
Great news. Florida, get on the ball. If a teacher is evaluated using the VAM measurement tool, isn’t there a way to use a VAM-style tool to evaluate the effectiveness of the company vending the system of evaluation in relationship to their performance in boosting student growth?
LikeLike
When I read the link, it seemed to emphasize that EVAAS is designed to “assess and predict student performance with precision and reliability.” They included a few endorsements from other states that are trying to dismantle public schools. They did not mention that the system can be used to evaluate teachers. Like every other state that has tried to evaluate teachers through test scores, Texas has had to face the wrath of railroaded teachers that file lawsuits. At least their union is standing up for aggrieved teachers. When the federal government is bribing districts to adopt VAM, many states take the bait. When the lawsuits arrive, more states will be inclined to drop this unscientific, misguided practice.http://vamboozled.com/houston-lawsuit-update-with-summary-of-expert-witnesses-key-findings-about-the-evaas-vam/
LikeLike
It’s worth noting that Terry Grier, who left as of March 1, was one of those touting that the PSAT predicted Advanced Placement test score.
Grier used to cite a College Board-funded “study” that suggested the PSAT-AP relationship. But, if read carefully, the study made clear that the PSAT population and the SAT population and the AP population cited in the study were, in fact, very different populations and that trying to extrapolate any relationships among and between them was fraught with error.
Grier hadn’t read the study.
Fairfax County, one of the biggest and “best” suburban school systems in the country hired Karen Garza as its new superintendent several years back. Before Fairfax, Garza was superintendent in Lubbock, Texas, and prior to that she was the Chief Academic Officer for Houston Independent School District. She was a top deputy to Terry Grier.
When Garza left Houston to become superintendent in Lubbock, the Houston district’s news release noted this:
“Dr. Garza led the development of HISD’s ASPIRE program, the school improvement effort that paid more than 18,000 teachers and instructional staff more than $70 million in performance bonuses over the last three years based on the academic improvement of children.”
Houston’s ASPIRE program was developed through a “quick and relatively noncollaborative planning process” that was not transparent and that relied on “a complex formula” that teachers did not understand. Worse, teachers “were not allowed access to the data that formed the basis of their performance awards.” The ASPIRE merit pay plan was funded (in part) by the Broad, Gates and Dell foundations, some of the top supporters of corporate-style education “reform.”
See, for example: http://www.cecr.ed.gov/guides/summaries/HoustonCaseSummary.pdf
Researchers and test experts caution against the use of value-added models to evaluate teachers. They note that “value-add models of teacher effectiveness are highly unstable,” and that “Teachers’ value-added ratings are significantly affected by differences in the students who are assigned to them.” Testing expert Jim Popham says that the use of student test scores to evaluate teachers “runs counter to the most important commandment of educational testing — the need for sufficient validity evidence.”
But Garza called value-added models “proven methodology” that are both “valid and reliable.” She said they offer up “increased accountability for schools, teachers and students.” Researchers say that “Value-added ratings cannot disentangle the many influences on student progress.” The National Research Council concluded that value-added models “should not used to make operational decisions because such estimates are far too unstable to be considered fair or reliable.” Yet Karen Garza said they can be used for “recognizing excellence,” for “informing practice,” and for “improving teaching and learning.” And while value-added models “use complex mathematics” that most educators – most people – do not understand to arrive at teacher evaluation scores, Garza said that value-added “reports are easy to interpret.”
Apparently she knows something that the experts do not.
I’ve said this a number of times on this blog: public education’s “leadership” has some very serious deficiencies.
LikeLiked by 1 person
Democracy,
Leaders should learn critical thinking skills
Otherwise they are sheep or parrots or lemmings
LikeLike
I hope that you can secure permission to publish this brief statement on your blog. It is a powerful statement about VAM from someone who is well- qualified to speak. The statement is far better than the AERA statement. It is a clear message that every federal, state, and local official should understand, along with teachers and others who are silently taking these evaluations as if they have some objectivity, validity, and reliability…
http://edr.sagepub.com/content/45/4/267.extract
Of course the bean counters will ask for an alternative. One of the best is to expand the opt out movement and put the schools boards, state officials, the federal officials and especially Congrees on the spot, for once.
Require them to justify their use of so-called growth measures, SLOs, SGOs, and VAM and stack ratings of students, teachers, schools, and states as if education is a perpetual game of being numero uno.
How about some celebrations of real achievements with students and teachers and parents all involved in determining those, how to represent them, and asserting why these achievements matter to specific students as well as the local community.
A radically individualized approach to evaluation might be a first step in purging the education of our students from the perpetual threats of tests and the casual acceptance of endless test preps as OK… rather than what these rituals are…a version of cheating and corruption too widely accepted as if perfectly normal.
How about ending the loss of time for teaching and learning caused by the prejudgment that accountability is only possible by the budget-busting demands of testing companies, and federal mandates.
Let us see and read the poems students have written, hear their songs and concerts, see their plays performed, be amazed by their practical uses of math, celebrate their ingenuity in creating and making perfectly wonderful images and things, even if “useless.” Stop the perpetual nagging about college and career ready, beginning in kindergarten.
How about honoring students as human beings and learners, theircapacity for caring for each other and for planet earth. How about allocating some time for public expressions of gratitude of these hallmarks of some generosity in spirit.
Narrow gauge measures that misrepresent so much that should be valued in education have created an aura of indisputable truth around the idea of an achievement gap that can be closed. Of course this gap, a statistical artifact, can never be closed period. Why? The gap is created by designing tests so that the scores are distributed as a bell curve, meaning (as everyone should know) that half of our students will be above average, half below average, in perpetuity, forever and forever. That is their statistical fate.
It is long past time to dump the assumption that there is only one way to be accountable for investments in public education.
LikeLike
From the link;
Steven Klees points out the truth about VAM/SLOs/SGOs:
The AERA Statement on the use of value-added models (VAM) is very welcome (American Educational Research Association, 2015) but only alludes to the principal problem with them — misspecification. The AERA statement argues that to “isolate the contributions of teachers and leaders to student learning” is “very difficult.”
It is not [difficult] — IT IS IMPOSSIBLE — even if all the technical requirements in the statement are met.
LikeLike
“VAM on the run” (with apologies to Sir Paul McCartney)
Well, the VAM exploded with a mighty crash
As it fell into the sun,
And the teacher said to the VAMmer there
“I hope you’re having fun.”
VAM on the run, VAM on the run.
And the Chetty man and Virginia ham
Were searching every one
For the VAM on the run,
VAM on the run
VAM on the run,
VAM on the run.
Well, the model maker drew a heavy sigh
Seeing VAM was dead and done
And a bell was ringing in the village school
For the VAMmers on the run.
VAM on the run,
VAM on the run.
And the Chetty man and Virginia ham
Were searching every one
For the VAM on the run,
VAM on the run
Yeah the VAM on the run,
VAM on the run
VAM on the run
VAM on the run
Well, the night was falling as Reformer world
Began to topple down.
In the town they’re searching for it everywhere
But it never will be found.
VAM on the run,
VAM on the run.
And Vergara judge who held a grudge
Will search for evermore
For the VAM on the run,
VAM on the run
VAM on the run
VAM on the run
LikeLiked by 1 person
$ 680K – let’s think about that……
LikeLike
They could have saved a lot of money by just flipping a coin to decide which teachers were “bad”.
It would have worked just as well, according to researcher Stuart Yeh, who performed “A Reanalysis of the Effects of Teacher Replacement Using Value-Added Modeling” by looking at a broad swath of VAM studies.
If they had the kids in the lower grades doing the flipping as a lesson in probability, it wouldn’t cost the school district anything.
And the kids would learn about random processes — like coin flipping and VAM as an added bonus.
LikeLike