Archives for category: Teacher Evaluations

Four years ago, I was in Colorado to discuss education policy. This was in the heady early days of Race to the Top (which Colorado did not win, despite its whole-hearted embrace of everything Arne Duncan wanted). On one occasion, I was scheduled to debate State Senator Michael Johnston, the darling of the “reform” crowd. Johnston had written a bill that was coming to a vote that very day. His bill made student test scores count for 50% of every educators’ evaluation. An effective evaluation, his bill decreed, required growth in student scores. Johnston called his bill something like “Great Schools, Great Educators.” Or something like that. Every bill these days must contain at least one impossible promise in its title.

As I said, we were supposed to debate in front of a packed room of civic leaders, maybe 80 or so people. I waited and waited. No Johnston. Finally, I got up and spoke my concerns in his absence. No sooner did I finish than the doors at the back of the room opened and out popped young Senator Johnston. I say young because he appeared to be about 25, though I think he was actually 32. He was then considered the leading voice of education reform in the Legislature, despite members who were retired and experienced educators. Senator Johnston had served two years in Teach for America, then was principal of a school for two years, then ran for state senate. And now he was rewriting the state’s education laws! Truly a whiz kid!

Since he did not hear me, he did not have to respond to anything I said. Instead, he spoke in glowing terms of his legislation. He had an almost mystical faith in the amazing results that would automatically materialize as soon as teachers and principals were evaluated by the academic growth of their students. He seemed to believe that the only source of low scores was the absence of incentives and sanctions for those unmotivated, possibly lazy educators. Everyone, it seemed, wanted to believe that he knew what he was talking about.

Now, we know it takes time to phase in new policies and practices. As Bill Gates famously said, “It will take a decade to know whether this stuff works.” What he meant by “this stuff,” I guess, is the idea that privatization and measuring teacher quality by student scores will make students better educated. My own view is that we should stop looking for the “secret sauce” because it is a chimera. Instead, we should do what we know works, which is reduced class sizes, early childhood education, family education, experienced teachers, healthy children, a full and rich curriculum, and the wraparound services that children need. But all that is complicated, not simple; our data-driven reformers like simple solutions, the bumper sticker ideas.

But surely we should see some positive movement in Colorado, don’t you think? And it should be cumulative, stronger every year as the “reforms” take hold.

The latest state scores from Colorado–which has been dominated by data-driven reformers for a decade– are unimpressive. Actually, the scores of third-graders, who have known nothing other than a testing culture, took a slight dip. In truth, they were flat.

Oh, well, maybe next year, we will see the miracle that Senator Johnston promised. Or the year after that.

Meanwhile Senator Johnston has been invited to be Alumni Commencement Speaker at Harvard Graduate School of Education, which has aroused some protest. This is allegedly a tribute to his great accomplishment in Colorado, where every year his promises grow more hollow. How many of the graduates at HGSE would want to work under Johnston’s law? Presumably, students at HGSE read research and know that VAM is Junk Science.

Laura Chapman says it is no improvement to substitute student growth in test scores for plain old test scores. Both reduce teaching and learning to multiple choice test questions.

She writes, in response to this post:

“Instead of judging schools solely by test scores, they might be judged–at least in part–by student growth.”
This is not an improvement of any kind, but the precise language from Race to the Top Legislation (see reference below).

In federal and state policy “student growth” is just a euphemism for a gain score from pre-test to post-test, or year-to-year. In other words, the term “growth” has been thoroughly corrupted to mean just another score, and preferably a score with properties that can be processed to produce a VAM–value added score. (See reference below on the new grammar…)

Do not be mislead. The marketeers of “growth” as if this is some gold standard or “fair” measure for judging educational activity are engaging in a propaganda campaign. Participants include USDE and its hired hands who know that this term “growth” has a rich and elaborate semantic reach in education. They are cynically trying to cut away understandings of growth and development as teachers understand it for individual students–multifaceted and asynchronous (e.g. bright but socially awkward; coordinated dancer, but not an athlete; enchanted with calligraphy but has terrible handwriting). To be sure, there are normative patterns for a large number of students, but so-called “developmental levels” also mask all of the wondrous variability in students. Forget all that, the new meaning of “growth” is a gain or increase in a metric derived from a test.

A perfect example of the marketing effort on behalf of redefining “human growth” (as a difference in metrics) is the infamous “Oak Tree Analogy” (see reference below)–that conveniently ignores that fact that students, unlike trees, have minds of their own.

I call this a cynical move because the oak tree analogy is framed to place teachers in the role of workers in a nursery in charge of providing the “nutrients” that are needed for trees to thrive. This frame, as Lakoff and Johnson remind us, taps a “nurturing parent” metaphor for teachers, and also the traditional role for women. The campaign to portray teachers as bad nurturers, lay, soft, uncaring is nowhere more evident that in the excessive use of “rigor” and “rigorous” as obligatory adjectives for almost everything bearing on “improvements” in education. See Lakoff, G., & Johnson, M. (2008). Metaphors we live by. University of Chicago press.

Repeat. Federal and state policy documents define “growth” as a gain in pre-test to post test scores, and a gain in year-to-year scores. Such scores are used to radically simplify judgments about districts, schools, teachers, and students. The distorted views of education produced by aggrandizing tests and “metrics” as if these refer to the actual complexities of human growth and development–perceptual, intellectual, social, physical, creative, aesthetic–is a fraud.

For federal language for “growth” see: Final Definitions 559751-52 Federal Register / Vol. 74, No. 221 / Wednesday, November 18, 2009 / Rules and Regulations DEPARTMENT OF EDUCATION 34 CFR Subtitle B, Chapter II [Docket ID ED–2009–OESE–0006]

RIN 1810–AB07 Race to the Top Fund AGENCY: Department of Education.Retrieved from http://www.gpo.gov/fdsys/pkg/FR-2009-11-18/pdf/E9-27426.pdf

For the false comparison of human development and oak tree “growth” see:

Value-Added Research Center. (2012). Teacher effectiveness initiative value-added training oak tree analogy. Madison: University of Wisconsin. Retrieved from Retrieved from http://varc.wceruw.org/tutorials/oak/index.htm

For the cynical promotion of a preferred “grammar” for education see:
Reform Support Network. (2012, December). Engaging educators, Toward a New grammar and framework for educator engagement. Author. Retrieved from http://www2.ed.gov/about/inits/ed/implementation-support-unit/tech-assist/engaging-educators.pdf

The centerpiece of Race to the Top is evaluating teachers by test scores. The students of good teachers, Arne Duncan and Barack Obama believe, get higher scores. If they have low scores, it is the fault of bad teachers. There was no evidence for their beliefs, other than the speculations of economists and statisticians. Real teachers never believed the theory, because they know that many favors affect test scores, not just teachers.

Thirty five states and DC followed Duncan’s lead, even though his hunch lacked any evidence . Lyndsey Layton has a comprehensive article in today’s Washington Post, describing the latest study to disprove Duncan’s theory.

Spurred on by Duncan, many states now use test scores to determine tenure and compensation. Duncan recently said he wants to judge the quality of teacher education programs by the test scores of students taught by their graduates.

Secretary Duncan’s love affair with standardized testing is inexplicable. There can be no question that he has caused immense damage to children, teachers, and public education.

Barbara Madeloni, who led the fight against outsourcing teacher credentialing to Pearson, was elected president of the Massachusetts Teacher Association, will take charge of a union of 110.000 educators

“Until last August, Madeloni directed the Secondary Teacher Education Program at the University of Massachusetts.

“While UMass said her employment ended as part of a move to reduce the use of adjunct professors, Madeloni stated in interviews that the school was punishing her for opposing a project in which UMass tested a teacher assessment program for the for-profit company Pearson.

“Madeloni, 57, said in an interview Sunday she plans as MTA president to “amplify the voice of educators and be a leader at the national level.”

“She noted that her victory comes amid efforts in Los Angeles, Seattle and Chicago to shift the debate back to supporting high-quality public education and the people who provide it over the interests of for-profit companies in the field.

“It should be national news,” Madeloni said of her win in Massachusetts. “It’s a message to everybody that teachers will not be silent and compliant as this assault on public education continues — and undermines public education. This is foundational to democracy and we need to defend it.”

Paul Karrer, who teaches fifth-grade in Castroville, California, takes a look at our policymakers’ obsession with bad teachers. Who are they? How can they be found out and fired?

Here is one example of a bad teacher:

“The Low Score bad teacher — Education reformers want high-stakes testing to be a prime determinant in teacher evaluation. But if one looks under the tests, interesting facts pop up. Often, teachers who were Teacher of the Year find they are considered bad teachers in the following years. How can this be? Because class composition changes. Teaching assignments (grade-level) change. And unlike charter schools, which expel obstructive, destructive and obnoxious kiddos — those in the public realm must teach all the kids. Just one grade-A-whack-a-mole angry student can destroy a classroom. Many teachers have many more than one. Such a child’s presence is subtractive to the learning environment of others.”

There are more.

But who is behind this pursuit and does it make sense?

“My point in all this is to show that variables — normal life variables — impact classroom outcomes. When pregnant teachers, their compassionate spouses and ill teachers are labeled as bad teachers, something is very, very wrong.

“The profit-oriented talking heads of education reform want to monetize public education. Ed reformers would have us believe poverty, trauma, parental drug use, violence, incarceration and homelessness have no impact. Teachers are losing their profession. Kids are losing their teachers. And communities are losing the democratic concept of public schooling.

“In the end, it is the wealthy profiteers and captains of privatization who are pointing fingers at hard-pressed teachers who work in communities of failure. It is a much easier political fix to scream “fire” than it is to acknowledge the conditions of poverty. And it makes money for a few too.”

Audrey Amrein-Beardsley noticed an interesting pattern among the states that won Race to the Top funding.

Most were states with highly inequitable school finance systems, as noted by the Education Law Center of New Jersey.

But Beardsley saw other correlations.

She writes:

“In this case, correlational analyses reveal that state-level policies that rely at least in part on VAMs are indeed more common in states that allocate less money than the national average for schooling as compared to the nation. More specifically, they are more likely found in states in which yearly per pupil expenditures are lower than the national average (as demonstrated in the aforementioned post). They are more likely found in states that have more centralized governments, rather than those with more powerful counties and districts as per local control. They are more likely to be found in more highly populated states and states with relatively larger populations of poor and racial and language minority students. And they are more likely to be found in red states in which residents predominantly vote for the Republican Party.”

These were the states most willing to evaluate teachers by test scores (VAM), despite the absence of evidence for doing so.

In Florida, teachers are given ratings based on the scores of students they never taught.

 

Teachers in several counties challenged the law in court.

 

The judge agreed that the system was unfair, but refused to overturn it.

 

Where teachers are concerned, Junk Science is just fine.

 

It is okay to rate a teacher based on the performance on tests of students the teacher never met, never taught.

 

This is “reform.” Thanks, Arne Duncan, for Race to the Top.

 

Thanks for introducing this insane, stupid policy into our nation’s schools.

 

The true education miracle will be if American public education can survive eight years of stupid policies like this one in Florida.

A federal judge in Florida dismissed a lawsuit against the state evaluation system, declaring that it was unfair to rate teachers based on the scores of students they never taught but not unconstitutional.

The evaluation system may be stupid; it may be irrational; it may be unfair; but it does not violate the Constitution. So says the judge.

An article in the Florida Education Association newsletter described the ruling:

“The federal lawsuit, known as Cook v. Stewart, was filed last year by the FEA, the National Education Association and seven accomplished teachers and the local education associations in Alachua, Escambia and Hernando counties. The lawsuit challenged the evaluation of teachers based on the standardized test scores of students they do not teach or from subjects they do not teach. They brought suit against the Florida commissioner of education, the State Board of Education and the school boards of those three counties, who have implemented the evaluation system to comply with 2011’s Senate Bill 736.

“On Tuesday afternoon, U.S. District Judge Mark Walker dismissed FEA’s challenges to the portions of SB 736 that call for teachers to be evaluated based upon students and/or subjects the teachers do not teach, though he expressed reservations on the practice.

We are disheartened by the judge’s ruling. Judge Walker acknowledged the many problems with this evaluation system, but he ruled that they did not meet the standard to be declared unconstitutional. We are evaluating what further steps we might take in this legal process.

Judge Walker indicated his discomfort with the evaluation process in his order.

“The unfairness of the evaluation system as implemented is not lost on this Court,” he wrote. “We have a teacher evaluation system in Florida that is supposed to measure the individual effectiveness of each teacher. But as the Plaintiffs have shown, the standards for evaluation differ significantly. FCAT teachers are being evaluated using an FCAT VAM that provides an individual measurement of a teacher’s contribution to student improvement in the subjects they teach.” He noted that the FCAT VAM has been applied to teachers whose students are tested in a subject that teacher does not teach and to teachers who are measured on students they have never taught, writing that “the FCAT VAM has been applied as a school-wide composite score that is the same for every teacher in the school. It does not contain any measure of student learning growth of the … teacher’s own students.”

In his ruling, Judge Walker indicated there were other problems.

“To make matters worse, the Legislature has mandated that teacher ratings be used to make important employment decisions such as pay, promotion, assignment, and retention,” he wrote. “Ratings affect a teacher’s professional reputation as well because they are made public — they have even been printed in the newspaper. Needless to say, this Court would be hard-pressed to find anyone who would find this evaluation system fair to non-FCAT teachers, let alone be willing to submit to a similar evaluation system.”

“This case, however, is not about the fairness of the evaluation system,” Walker wrote. “The standard of review is not whether the evaluation policies are good or bad, wise or unwise; but whether the evaluation policies are rational within the meaning of the law. The legal standard for invalidating legislative acts on substantive due process and equal protection grounds looks only to whether there is a conceivable rational basis to support them,” even though this basis might be “unsupported by evidence or empirical data.”

Audrey Amrein-Beardsley has been consulting with the seven Houston teachers who filed a lawsuit in federal court against the use of value-added metrics in their evaluations.

 

She has conducted extensive VAM research in Houston and concluded it was arbitrary and inaccurate. “Houston, the 7th largest urban district in the country, is widely recognized for its (inappropriate) using of the EVAAS for more consequential decision-making purposes (e.g., teacher merit pay and in the case of this article, teacher termination) more than anywhere else in the nation.”

 

She believes that this is the lawsuit that has the potential to bring down VAM as a valid way of measuring teacher quality.

 

Read here to learn why.

 

If VAM goes down, as it should, it would be yet one more piece of evidence that Race to the Top is a $5 billion flop, as if any more evidence were needed.

 

Of course, even a court victory against inappropriate teacher evaluation would not deter our Secretary of Education from claiming victory. If he were on the basketball court, he would claim victory if his team were beaten 152-18; we would never hear the end of those heroic, astonishing, incredible 18 points.

 

Jonathan Pelto here reports on a great new piece by civil rights lawyer Wendy Lecker.

 

He writes:

“In her latest MUST READ commentary piece, fellow public education advocate, Wendy Lecker, lays out the facts about Governor Malloy’s unfair, inappropriate and fatally flawed teacher evaluation system. Like the junk bonds that helped take down Wall Street, Connecticut’s teacher evaluation system is based on junk science and false assumptions.

 

The question is not whether the state should have a comprehensive teacher evaluation system, but whether the corporate education reform industry will continue to stand in the way of developing one.

 

Lecker says that Governor Dannell Malloy’s teacher evaluation system is fundamentally flawed.

 

Lecker writes that the solution to failed tests is not more tests.

 

From her article:

 

Fact: Connecticut’s teacher evaluation plan, because it relies on student standardized test scores, is fundamentally flawed. Student test scores cannot measure a teacher’s contribution to student learning. In fact, the president of the Educational Testing Service recently called evaluation systems based on student test scores “bad science.”

 

Rather than admit failure, the Malloy administration is trying futilely to “fix” the fatal flaw. Last week, PEAC, the panel charged with developing Connecticut’s teacher evaluation system, working under the direction of Commissioner Stefan Pryor, approved a change which calls for more standardized tests to be included in a teacher’s evaluation.

 

The commissioner’s “solution” is to add interim tests to a teacher’s rating. Determining what tests will be used, how they will be aligned to the standardized tests, and how all the test scores will be rolled into one “score” for teachers, will likely render this change completely unworkable.

 

She adds:

 

A recent comprehensive study by Northwestern Professor Kirabo Jackson found that children with teachers who help them develop non-cognitive skills have much better outcomes than those who have teachers who may help them raise test scores. Jackson found that every standard deviation increase in non-cognitive skills corresponds to a significant decrease in the drop-out risk and increased rates of high school graduation. By contrast, one standard deviation increase in standardized test scores has a very weak, often non-existent, relationship to these outcomes. Test scores also predict less than 2 percent of the variability in absences and suspensions, and under 10 percent of the variability in on-time grade progression, for example.

Increases in non-cognitive abilities are also strongly correlated with other adult outcomes, such as a lower likelihood of arrest, a higher rate of employment and higher earnings. Increased test scores are not.

In short, focusing on non-cognitive abilities, those not measured by test scores, are more important in predicting success in high school and beyond.

 

Why are the corporate reformers so wedded to standardized tests that they themselves probably could not pass? They love data. They want Big Data. They believe that every problem can be solved by measurement and manipulation of Big Data. They also believe that they can create the appearance of “failing public schools” by generating data showing how many kids are not meeting an artificial benchmark. This enables them to argue for more charter schools that are free to exclude the children who did not meet the artificial bench mark. Big Data is now part of the tool kit of privatization. It is not about helping kids or improving education, but finding a rationale for turning public dollars over to private managers. If we really wanted to help kids and improve education, we would take the billions now going into testing and use it to reduce class sizes, to increase the arts, and to provide every child the medical care they need.