Audrey Amrein-Beardsley testified on behalf of the plaintiffs (the teachers) in the court case against New Mexico’s teacher evaluation system. She is an expert on teacher evaluation and has had the benefit of having been a teacher. Her blog “Vamboozled” regularly criticizes the misuse of test-based evaluations programs (like VAM, value-added measurement) that use the rise or fall of student test scores as their measure of teacher effectiveness.
In this post, she gives an overview of day three of the trial. The main “expert witness” for the state, testifying in favor of VAM, was Tom Kane of Harvard. He previously directed the Gates Foundations MET (Measures of Teacher Effectiveness) study, which promoted the use of VAM.
It is noteworthy that neither Beardsley nor Kane was able to analyze New Mexico’s data because the state did not release them or make them available, even to its own “expert witness.”
Kane admitted that he
had not examined any of New Mexico’s actual data. This was surprising in the sense that he was actually retained by the state, and his lawyers could have much more likely, and literally, handed him the state’s dataset as their “expert witness,” likely regardless of the procedures and security measures (but perhaps not the timeline) I mentioned prior. Also surprising was that Kane had clearly not examined any of the exhibits submitted for this case, by both the plaintiffs and the defense, either. He was essentially in attendance, on behalf of the state, to “only speak to [teacher] evaluations in general.” As per an article this morning in The Albuquerque Journal, as an “expert witness” he “stressed that numerous studies show that teachers make a big impact on student success,” expressly contradicting the American Statistical Association (ASA), while referencing studies of primarily his econ-friends (e.g. Raj Chetty, Eric Hanushek, Doug Staiger) and those of his own (e.g, as per his Measures of Effective Teaching (MET) studies), although this latter (unambiguous) assertion was not highlighted in this article. For more information in general, though, see the articles this morning in both The Albuquerque Journal and The Santa Fe New Mexican.
Then the state called the superintendent of the Roswell Independent School District to testify in favor of the state’s evaluation model. He said that the new system was an improvement over the old one. He also testified that he would not use the ratings to fire teachers, because he already had a teacher shortage. He told the local newspaper that:
“I am down teachers. I don’t need teachers, number 1, quitting over this and, number 2, I am not going to be firing teachers over this.” His district of about 600 teachers currently has approximately 30 open teaching positions, “an unusually high number;” hence, “he would rather work with his current staff than bring on substitutes” in compliance. So while he testified on behalf of the state, he also testified he was not necessarily in favor of the consequences being attached to the state’s teacher evaluation output, even if as currently being positioned by the defense as “low-stakes.”
Diane Ravitch's blog 
“The night they drove Statricksy down” (parody of “The night they drove Old Dixie down)
Thomas Kane is my name and I drove on the VAMville train
‘Til Audrey Beardsley came and tore up the tracks again.
After the ASA* paper knife , we were hungry, just barely alive.**
By twenty-fourteen, Rich man had fell.
It’s a time I remember, oh so well.
(*American Statistical Association, **only had $ 45 million from the “Rich man”, Bill Gates)
The night they drove statricksy down
And all the bells were ringing,
The night they drove statricksy down
And all the people were singing
They went, “Na,na,na.na,
Na na na na na na na na na.”
Back with my colleague, Raj Chet-ty, when one day he called to me,
“Thomas, quick, come see, there goes the Gastesly Billee!”
Now I don’t mind I’m choppin’ stats, and I don’t care if I’m paid by the brats
You take what you need and leave the rest,
But they should never have taken the VAMmy best.
The night they drove statricksy down
And all the bells were ringing,
The night they drove statricksy down
And all the people were singing
They went, “Na,na,na.na,
Na na na na na na na na na.”
Like my “Father”*** before me, I will work the VAM
And like my colleague before me, I took a junk-stat stand.
He was just 34, proud and brave,
but the ASA put him in his grave.
I swear by the mud below my feet
You can’t raise a Kane back up when he’s in defeat
(***Eric Hanushek, Father of VAM)
The night they drove statricksy down
And all the bells were ringing,
The night they drove statricksy down
And all the people were singing
They went, “Na,na,na.na,
Na na na na na na na na na.”
The night they drove statricksy down
And all the bells were ringing,
The night they drove statricksy down
And all the people were singing
They went, “Na,na,na.na,
Na na na na na na na na na.”
LikeLike
Even Judge Judy makes them produce the paperwork. Only the dimwits show up with nothing. And then she mocks them and tells them they lose.
LikeLike
Makes sense to me. If Kane were to analyze the New Mexico data and testify about his conclusions, the state would have to disclose the data to plaintiffs to permit them to rebut Kane’s conclusions.
LikeLike
Why weren’t the data available during discovery?
LikeLike
I don’t know. Plaintiffs might not have asked for the data. Or maybe they asked and the state objected and the dispute was resolved in the state’s favor. No way to know from where I’m sitting.
LikeLike
Would you guess that the plaintiffs requested it? Or did they know it would play out in this embarrassing way?
The federal NCLB Waiver Agenda was based on an unconstitutional law (NCLB).
Any sane and rational judge who hears what has been inflicted in the public schools of this country could not possibly rule in favor of the USDOE or state ed departments.
LikeLike
I would assume plaintiffs requested it, and I would assume the state fought the request. Could have played out any number of ways. Sometimes a party effectively runs out the clock on an adversary’s discovery requests, and the adversary will abandon requests simply because the clock’s run out. Totally speculating on this. I don’t get the sense that the fact that Kane didn’t review the state’s data is embarrassing, though. If plaintiffs haven’t presented their expert analysis of that data in their case in chief, I don’t see why the state would feel the need to present its own expert analysis of the data. All it would do is open the door to a new line of argument for plaintiffs.
LikeLike
Maybe it was – but nobody understood it!
LikeLike
The Roswell Superintendent’s promises are irrelevant. That’s the best NM could do?
I’m getting the sense that the whole VAM scam is falling apart at the seams. This is just the start of the legal onslaught that will be unleashed by teachers across the country. The “No Attorney Left Unemployed Act”.
LikeLike
I cannot actually believe that I predicted this principal’s reaction. Based on my limited statistical knowledge and extensive experience with test construction and execution, I could see how it would play out. Teacher shortage means work with what people you can hire, etc.
Those of us old enough remember when districts had to hire warm bodies. It won’t matter what kind of school you are running–public, private, parochial–you take what you can get. He who pays the best and provides the greatest benefit wins the bodies, good or bad. This will be true for charters as well. As the public is for almost no taxes in some states, the boards will be hiring anybody breathing (Las Vegas) and for-profit charters may have problems even with their real estate deals.
Been there. Done that. 1969 NYC. Worked with men teaching to escape the draft who were law students by night. Worked with vets as they were rated in higher but who were not always competent.
We joked about the irrationality if the system. Now the best and brightest (in their own minds) are recreating it.
LikeLike