Data hounds continue to search for a measuring stick to identify teacher quality.
They can’t believe they are on a fruitless hunt, like trying to find a barometer or yardstick to say which piece of art is best, which doctor is best, which…… as though human judgment means nothing.
Here is Matt Di Carlo summarizing the research on the instability of VAM, meaning that the best teacher this year might be only average next year, or vice versa.
A little known group called Educators for Shared Accountability designed a rubric for evaluating Secretaries of Education. It incorporates multiple measures.
By its metric, Richard Riley was our best national leader.
Check out Secretary Duncan’s value added rating.
The New York City teacher evaluations were released, and there was nearly no media coverage.
Mayor Bloomberg noticed. Ad Peter Goodman points out on his blog,
“The mayor didn’t like the original law, didn’t like the law which protected teachers from the public release of the scores and doesn’t like the requirement that the details of the plan must be negotiated with the collective bargaining agent, the union.
“On his weekly radio program he made it clear – he has no intention of negotiating a plan – he’ll accept the $250 million cut in state funding unless the union succumbs to all his preconditions. Apparently he “forgot” that the current law prohibits the release of the scores.”
Goodman checked with principals and teachers and they seemed genuinely puzzled by the ratings.
They don’t know what they mean or how they are supposed to help.
“UFT President Mulgrew announced that 6% of teachers were rated “ineffective” and 9% rated “highly effectively.” In order to be charged a teacher must be rated “ineffective” on their overall score or on the VAM and “locally negotiated” section for two consecutive years. When we consider the “instability” of the scores – wide year to year variation – the percentage of teachers impacted will be quite low.”
So very few teachers will be found ineffective, and anyone who is discharged on the basis of these flawed metrics is likely to sue.
Think of the hundreds of millions wasted on this junk science and how the money might have been used to improve schools.
Linda Darling-Hammond and Edward Haertel of Stanford University explain why value-added assessment doesn’t work and how inaccurate it is.
Will John Deasy listen? Will the Gates Foundation listen?
Will the Los Angeles Times, which published their article, stop seeking names to publish inaccurate data about teacher “effectiveness”?
The Los Angeles Times (!) has an outstanding article by reporter Teresa Watanabe about the new teacher evaluation system. It is based on growth in test scores and on computer modeling. The focus is on one teacher who seems to do all the right things: last year, he got a good rating but not this year. What changed? Nothing.
The United Teachers of Los Angeles has been fighting the LAUSD’s efforts to impose this flawed system on all teachers.
Eventually, after we have spent billions of dollars on these mechanical systems, the policymakers will figure out that the experts were right: the ratings reflect who is taught, not teacher quality.
Remember: no other nation in the world is judging teacher quality this way. This is our own nutty idea. It’s main accomplishment: demoralization of teachers.
Poor Tom Friedman! Everyone who knows anything at all about education knows that Tom has egg all over his face. They are either angry at him or laughing at him. He made such a fool of himself with his over-the-top (the same one we are racing to) praise of Race to the Top. If he had ever talked to a real educator, he would have not have praised Race to the Top. Instead, he would have written about Libya or Syria. But, no, he chose to act like Arne Duncan’s PR flack, repeating Arne’s favorite lines and doing no fact-checking.
Fortunately we have EduShyster, who has done the fact-checking. The result of this laborious activity is that E.S. is worried that both Tom Friedman and Arne Duncan have extremely low value-added scores. Before long, both may be replaced by someone young, innovative, and data-driven.
From a reader in Maine:
I’ve always thought that teachers should suggest a better approach–Getting a percentage of their students’ incomes after high school graduation. The logic is that teachers who add value will produce students who make more money; therefore, to align incentives property, the teachers should get a cut of the incomes from their graduates.
Just think how much Steve Jobs’s estate owes California’s teachers! 
If you add the scores on standardized tests for five years in a row, can you tell who the best and worst teachers are?
No.
But that’s the theory behind value-added assessment.
The idea is that an “effective” teacher raises test scores every year. The computer predicts what the test scores are supposed to be, and the teacher who meets the target is great, while the one who doesn’t is ineffective and should be shunned or banished.
But study after study shows that value-added assessment is rife with error. As this paper from the National Academy of Education and the American Educational Research Association shows, value-added assessment is unstable, inaccurate and unreliable. Teachers who get high ratings one year may get low ratings the next year. Teachers are misidentified. Data are missing. The scores say more about which students were in the classroom than the teachers’ “quality” and ability to teach well.
Teachers of the gifted are in trouble because the students are so close to the ceiling that it is very difficult to “make” them get higher scores.
Teachers of special education are in trouble because their students have many problems taking a standardized assessment. A teacher wrote me last year to tell me that her students would cry, hide under their desks, and react with rage; one tore up the test and ate the paper.
Teachers of English language learners are in trouble because many of their students don’t know how to read English.
A superintendent in Connecticut wrote me to say that his state department of education is pushing the Gates’ MET approach. I urged him to read Jesse Rothstein’s critique. In fact, the MET study won the National Education Policy Center’s Bunkum award for research that reached a conclusion that was the opposite of its own evidence.
For a fast and accurate summary of what research says about value-added assessment, read this article by Linda Darling-Hammond.
VAM is junk science. Bunk science.
Just another club with which to knock teachers, wielded by those who could never last five minutes in a classroom.
Many people assume that value-added assessment started with Race to the Top.
Value-added assessment or value-added modeling means judging teachers by how much students scores went up.
Actually, it started in the 1980s, when William Sanders, an agricultural statistician in Tennessee, claimed that it was possible to measure student growth the way he was accustomed to measure the growth of plants, with the teacher as the independent variable.
In Dallas, at about the same time, a group of school district statisticians developed their own model to measure teacher effectiveness.
You would think that by now Tennessee and Dallas would be leading the nation, having figured out this stuff that the Obama administration has imposed on the nation. But they are not.
New York City started experimenting with a value-added model not long after Bloomberg took control. Marc Epstein, then a teacher at Jamaica High School, figured out that what the city was doing was shifting responsibility for learning from the student to the teacher. It seemed benign at the time. Now we can see this idea sweeping the nation, demoralizing teachers and turning schooling into a data-driven environment where learning becomes a numbers game. Anyone can play.
Marc, who holds a Ph.D. in Japanese naval history, is now a member of the large group of teachers in New York City called ATR (absent teacher reserve). His school was closed, through no fault of his own or any other faculty member. So with his long experience and deep knowledge of history, he floats from school to school. He is too expensive. A school can hire two young teachers in place of his salary. New York City’s Department of Education would prefer to keep teachers like him as ATR–collecting a salary without a real assignment–because…sorry, I can’t recall the reason. Maybe they hope he will go away, along with the hundreds or thousands of other teachers that have been displaced by a policy of closing schools and allowing new schools to maximize their budget by excluding veteran teachers.
Diana Senechal is having fun with the metrics business. In this post, she manages to combine data mania with the consumer-driven mentality of our current education policymakers.
All in one short article, you get a free parody of choice, value-added assessment, competition, stack ranking, and the Broad Superintendents Academy.
