Audrey Amrein Beardsley invited an economist to review Raj Chetty & Co’s effort to take down the statement of the American Statistical Assosociation.
Chetty and friends are the leading advocates for using test scores to rank teachers and fire those whose students have the lowest scores. The ASA report was inconvenient for their thesis, as it pointed out that teachers account for a small percentage of the variance in test scores, that what is observed is likely to be correlation all, not causal, and that putting so much weight on test scores was likely to cause bad effects.
There is also the inconvenient fact that VAM is no longer a neat theory, but has moved into the realm of reality. Some districts have used it, often with unfortunate results. One would think that academics would feel some obligation to see how their theory is working in practice rather than continue to market it on grounds of its theoretical elegance.
Diane Ravitch's blog 
I might understand their theory if these “rankings” weren’t based upon tests that are used to “measure” student “success” on “standards” for which many , if not most, teachers have had little to no experience in delivery of the new spin on teaching/learning.
To insert these tests into schools across the nation without respect to their local curricula and infrastructure, including student exposure to keyboards and use of software or testing programs, is simply absurd. To force districts to invest in computers, tech coordinators, and programs instead of certified, licensed teachers is detrimental to local programs and careers.
If there were some type of fair, local VAM, it might have some merit. Even then, it would be insufficient and misguided. The fact that there are some students who simply do not care about getting a diploma, and the truths of poverty are ever-present (I am aware of the exceptions, such as the homeless boy who is valedictorian) speaks to the inaccuracy of across the board VAM. Kids who don’t care DO exist. They may regret it later, but some don’t care. They may “blame” the teacher(s), but this impacts all that goes into the so-called VAM.
The whole mess is poorly thought out, cruel, and unfair.
LikeLike
It is a nightmare at our district. Even when the principal uses the rubrics provided, she is unfair in that part of the evaluation. She makes unfounded assumptions both positively and negatively. With VAM in place and our school’s need to focus on the lower students, to help them pass the test, sometimes the upper kids don’t make a gain that is meaningful. Some “driven” kids get so nervous that they erase everything and lower their scores. It is just not a science! But it can make life miserable for teachers. VAM makes teachers teach to the test. Before that, we taught with confidence, not practicing for the tests at every opportunity.
LikeLike
“One would think that academics would feel some obligation to see how their theory is working in practice rather than continue to market it on grounds of its theoretical elegance.”
Elegantly & eloquently stated. Economists and financial wizards who crashed our economy in 2008 employed computerized, theoretical mathematical algorithms to their fraudulent credit default swaps & mortgage bundling. No one in the years leading up to the crash clearly understood the implications of applying their complex formulae to the economy. The more complex the greater the excitement generated from in this band of elites.
It was only after millions of people lost their life savings and the banks went broke that theses same financial wizards & economists said “oops”.
It seems Chetty et al, have learned nothing. They are either willfully blind or will not admit they are out of their area of expertise.
LikeLike
It seems like there is a lot of very credible studies and professional opinions about how unreliable VAM is. Since it is being used with so much emphasis to evaluate teachers in a way that critically effects their income and employment, why isn’t there more class-action law suits, and litigation in general-particularly from teacher unions?
LikeLike
The teachers’ unions have bought into corporate reform. We will get NO help from them.
LikeLike
Not necessarily true. Our state (WA) and local unions were pretty united against Arne Duncan’s demand to change our legislation to say MUST use standardized test scores as part of the teacher eval. Currently it says CAN use. Quite a few of our admins and superintendents spoke out against it too. As for NEA – yes, they have bought into the corporate reform BS, but our locals pay little attention to them, and when the state union starts trotting out some of that NEA-corp mentality, they do frequently get a lot of pushback from the locals. I suspect that if Duncan attempts to punish us beyond the NCLB waiver loss – which apparently hasn’t been enforced yet? – there will be some pushback in the form of a lawsuit against the Feds.
LikeLike
It always amazes me how often the local unions are so in tune with their students and communities and how far removed from reality the NEA can be.
Our local group voted NI on participating in RttT. However, our superintendent wanted the few dollars we’d receive from it. (It was truly small.) He went on record in a big statewide newspaper saying that the “teachers just didn’t want to be evaluated” and the union leaders backed down. We have different leaders now, but it is too late to turn back the clock. We didn’t want the new evaluation system. True. But it was because we have a principal that simply makes up whatever she wants on her evaluations. The good as well as the bad comments are inaccurate. She even tells teachers she will “bury” them. That part is true. Hostile work environments are unpleasant, especially during evaluations. The common core was shoved onto us through classes called CORE, but we were told we were filling state standards at the time. No mention of CCSS was made until 2012. We were ‘training” from 2007 on.
LikeLike
In many states the laws relating to the use of VAM have been written in such a way as to pre-empt legal challenge at the state level. It is a condition of the employment contract. In some states the unions even signed on, hoping to gain from this concession and also hoped to be able to modify the VAM process later.
LikeLike
Interesting… but if VAM is as flawed and unjust, as it seems to be, there must be some legal rights for the victims of this bad policy.
LikeLike
What about a class action against Chetty and his friends?
LikeLike
A class action lawsuit against the author of a research article? I am not sure how that would work.
LikeLike
For the legal hurdles, think big tobacco.
LikeLike