Archives for category: Teacher Evaluations

Adam Urbanski, president of the Rochester, Néw York, teachers’ union, is struggling to make sense of the state’s teacher and principal evaluation system, which varies wildly from district to district. Scarsdale, perhaps the most affluent and high-scoring district in the state, had no “highly effective” teachers. But Rochester, one of the districts with high poverty and low scores, had many. The reality is that none of the formulas for reducing teaching to a number make any sense. Teaching is an art, a craft, and a bit of science. A great teacher may be great one year, not the next, or great with this class but not another. (APPR in Néw York is the Annual Professional Performance Review.)

The ratings in Néw York are referred to as HEDI: Highly Effective, Effective, Developing, Ineffective. A commenter on the blog recently said that “Developing” is considered a low grade but she hoped that she was “developing” every day as a teacher.

This is what Adam wrote to his members:

“The Rochester Miracle?”

“Each year, we re-negotiate our APPR agreement with the District to do all we can to make it less damaging to our student and more fair to teachers.

“We are making progress in reducing the number of Rochester teachers (be)rated as Developing or Ineffective (40% in 2012-2013 but 11% in 2013-2014) and increasing the number rated as Effective or Highly Effective (60% in 2012-2013 but 89% in 2013-2014). Just one year ago, only 2% of Rochester teachers were rated as Highly Effective. This year, that number increased to 46%.

“Why such a huge fluctuation? Maybe it’s because we re-negotiated the agreement; or because teachers set more realistic SLO targets; or because the NYS Education Department adjusted the cut scores in ELA and Math; or because huge fluctuations are typical of invalid and unreliable evaluation schemes. Who knows? In any event, we continue to press for the total abolishments of APPR.

“Meanwhile, we are negotiating a successor agreement that would further diminish excessive testing of students and wrongful rating of teachers.”

Our regular reader and commenter Laura Chapman offers us another nugget of informed analysis and wisdom:

She writes:

A press release dated NEW YORK, Oct. 28, 2013 /PRNewswire/ announced that The Leona M. and Harry B. Helmsley Charitable Trust was investing $3 million “to establish a rigorous research project to modify and align the Framework for Teaching with the Common Core State Standards (CCSS).

This project will happen in four districts. One of these (unnamed) is in NY state.

You can find the application to market the 2013 Danielson Framework in NY state at http://usny.nysed.gov/rttt/teachers-leaders/practicerubrics/Docs/danielson-application.pdf

There you will see that the application required empirical evidence in support of “each rubric.” Whatever that “each rubric” meant, the application was approved with very brief references to eight “empirical” studies, three with more elaborate descriptions of the methodology.

In addition to the questions I asked about the full spectrum applicability of the Danielson protocol, I should have asked about studies that paid attention to the “demographics” in the classrooms observed—the proportional composition of students who qualify for lunch programs, those in gifted programs, special education, students still learning English, recent transfers, and so on.

Every teacher knows how these distributions shift from class to class and make a huge daily difference in what is taught, how, and so on.

For a recent summary of the many problems with this and related high stakes evaluation schemes see Leading via Teacher Evaluation: The Case of the Missing Clothes?
(July, 2013) Joseph Murphy, Philip Hallinger and Ronald H. Heck

Click to access Teacher%20Eval%20-%20Case%20of%20Missing%20Clothes%20-%20Murphy.pdf

See also a 2014 VIP article by David C. Berliner in Teachers College Record. His online summary of the craze to evaluate teachers by flawed methods closes with this great sentence:

“In fact, the belief that there are thousands of consistently inadequate teachers may be like the search for welfare queens and disability scam artists—more sensationalism than it is reality.” http://www.tcrecord.org/Content.asp?ContentId=17293

EduShyster has been trying to crack the case of the mysterious disappearance of minority teachers in urban districts. She takes a close look at Boston, especially the prestigious Boston Latin School.

Here is her beginning:

“Today’s high-stakes question involves the demographics of our nation’s teaching force. When and where is it appropriate to discuss the urgent need to diversify the nation’s teaching force whilst failing to acknowledge what’s happening to the ranks of minority teachers who are already teaching? The answer: in whatever city Arne Duncan’s *bigger rigor* bus tour happens to have landed. You see, even as a much-needed conversation about the vital importance of having teachers of color in front of an increasingly diverse student body is taking place, a bouquet of reform policies is effectively pushing out existing teachers of color. Bundle up reader, because we’re headed to Boston where the nip of fall is in the air and minority teachers are being *reformed* right out of the city’s public schools.”

Read her links by opening her sad tale of the use of new evaluation systems to screen out teachers of color.

Audrey Amrein-Beardsley here summarizes and comments on a very enlightening interview with Jesse Rothstein in the Washington Post. Rothstein, an economist, conducts research on teacher evaluation and accountability.

Rothstein, on teacher evaluation:

“In terms of evaluating teachers, “[t]here’s no perfect method. I think there are lots of methods that give you some information, and there are lots of problems with any method. I think there’s been a tendency in thinking about methods to prioritize cheap methods over methods that might be more expensive. In particular, there’s been a tendency to prioritize statistical computations based on student test scores, because all you need is one statistician and the test score data….

“Why the interest in value-added? “I think that’s a complicated question. It seems scientific, in a way that other methods don’t. Partly it has to do with the fact that it’s cheap, and it seems like an easy answer.”

“What about the fantabulous study Raj Chetty and his Harvard colleagues (Friedman and Rockoff) conducted about teachers’ value-added (which has been the source of many prior posts herein)? “I don’t think anybody disputes that good teachers are important, that teachers matter. I have some methodological concerns about that study, but in any case, even if you take it at face value, what it tells you is that higher value-added teachers’ students earn more on average.”

“What are the alternatives? “We could double teachers’ salaries. I’m not joking about that. The standard way that you make a profession a prestigious, desirable profession, is you pay people enough to make it attractive. The fact that that doesn’t even enter the conversation tells you something about what’s wrong with the conversation around these topics. I could see an argument that says it’s just not worth it, that it would cost too much. The fact that nobody even asks the question tells me that people are only willing to consider cheap solutions.”

“Rothstein, on teacher tenure:

“Even if you give the principal the freedom to fire lots of teachers, they won’t do it very often, because they know the alternative is worse.” The alternative being replacing an ineffective teacher by an even less effective teacher. Contrary to what is oft-assumed, high qualified teachers are not knocking down the doors to teach in such schools.

“Teacher tenure is “really a red herring” in the sense that debating tenure ultimately misleads and distracts others from the more relevant and important issues at hand (e.g., recruiting strong teachers into such schools). Tenure “just doesn’t matter that much. If you got rid of tenure, you would find that the principals don’t really fire very many people anyway” (see also point above).

Peter Greene has an endless willingness to read the steady deluge of think-tank reports on how to fix teaching, how to fix schools, etc. it is not necessary to be a teacher to write these reports. That’s what think tanks do.

In this instance, he has read and dissected TNTP’s new report on how to fix tenure. Bear in mind that the original name of the organization, founded by Michelle Rhee (some claim that it was actually founded by Wendy Kopp but what difference), was The New Teacher Project. Its purpose was to place new teachers in urban districts. Thus, TNTP has a vested interest in teacher turnover as it creates more slots for its recruits to fill.

Given that anywhere from 40-50% of teachers don’t last five years, there are already plenty of slots anyway. One would think that a genuine reform would focus on how to recruit, support, and retain excellent teachers who want to make a career of teaching. But no, we still live inn an era when reformers are obsessed with he idea that schools are granting tenure too easily, and tenured teachers are in need of constant watch, lest they slip into their lazy, slacker habits bred of complacency.

Recommendation number one of the report is that no one should get tenure in less than five years. Greene says that any administrator who needs five years to decide whether a teacher is worthy of due process is a dope. (My word, not his.) it is also suggested that tenure be revocable based on test scores, which means it is not tenure at all.

Ken Futernick, a wise educator who has written about the improvement of the teaching profession for many years, has a brilliant article in the Los Angeles Times about “grand bargain” post-Vergara. Futernick testified for the state in the Vergara trial. He has long understood that schools in urban districts with low scores often have poor working conditions, inadequate resources, and high teacher turnover.

The term “grand bargain” typically refers to compromises by warring parties. In this case, he has laid out a program that all states can learn from.

He writes:

“Unless it’s overturned on appeal, the Los Angeles Superior Court’s June decision in Vergara vs. California making it much easier to fire teachers will hurt students if lawmakers, unions and other state education leaders don’t move beyond its limited focus and address the many factors that adversely affect student learning and teacher performance.

“Stakeholders must come together around a “grand bargain” that would address not only teacher incompetence but all the obstacles educators face that, in the end, prevent many students from learning.”

Making it easier to fire “bad teachers” won’t make it easier to hire good ones.

“To be sure, many of those who teach in poor neighborhoods don’t have the same effect on test scores as those who teach in wealthier schools. But most schools that serve poor and minority students — those with high concentrations of English learners, transient students, students with health problems and so on — have fewer resources to meet students’ many needs, larger class sizes and inadequate materials and facilities. In addition, they are staffed with many beginning teachers who turn over at high rates. Not surprisingly, student achievement suffers.

“Also, schools that serve poor students routinely assign teachers to subjects in which they have no expertise. For instance, a 2008 study showed that 27% of math courses in schools serving poor students were taught by teachers who were not qualified to teach math.

“Why are schools that serve poor and minority students overstaffed with inexperienced and out-of-field teachers? Most teachers seek to make a difference and are eager to teach disadvantaged students. But many don’t want to teach in such schools because most of them are extraordinarily difficult, dysfunctional places to work. The teachers there suffer from poor professional support, low morale, run-down facilities, a revolving door of principals and unrelenting accountability pressures.

“Ineffectiveness in the classroom often does not derive from incompetence.

“Consequently, administrators in these schools can’t attract and keep enough well-qualified, experienced teachers. That, in turn, highlights another critical flaw in the judge’s decision — the assumption that these schools can find suitable replacements for fired teachers. Quite the contrary, and administrators’ power to fire teachers without real due process will only exacerbate the teacher recruitment problem….

“For starters, the state should develop a new teacher dismissal process that is fair and efficient. It should not take years and hundreds of thousands of dollars to fire an ineffective teacher if he or she has been given a reasonable chance to improve, has been carefully evaluated and hasn’t done better.

“[Governor Jerry] Brown signed legislation this year that provides a fair and efficient way to adjudicate cases of gross teacher misconduct. Education leaders should develop a similar way to handle cases of teacher incompetence. They also should develop solutions for the other statutes that the court struck down, such as the one that allowed teachers with more seniority to keep their jobs during layoffs. California could do what other states have done, recognize experience along with other factors in making layoff decisions.

“But California must have a solid due process system for teachers, and contrary to popular belief, that’s all that tenure provides. Without a reliable way to determine whether a teacher is truly incompetent, the state will return to an era when employment decisions were fraught with abuse that included higher-salaried, experienced teachers replaced with less-expensive beginners and competent teachers fired because of their political or religious views.”

“Here is the framework Futernick suggests for a “grand bargain”:

“*The state must develop a robust teacher evaluation framework designed to help all teachers improve, not just to identify low performers. Such systems would ensure that principals and other evaluators have the time and training needed to conduct meaningful evaluations.

“*The state should build on the successful peer assistance and review programs that exist in places such as Poway Unified and San Juan Unified. These programs provide high-quality support to struggling teachers. Most participating teachers improve; those who don’t either leave voluntarily or are dismissed without grievances and expensive lawsuits.

“*The state and school districts must improve the conditions in hard-to-staff schools to attract and retain the best teaching candidates and the strongest principals. Among other things, these schools need high-quality professional development, time for teachers to plan and collaborate, and the authority to make professional decisions.”

Without adequate resources, changes in the law will be a hollow promise.

The following was reported at politico.com:

“AMERICANS CALL FOR STEPPING UP THE TEACHING PROFESSION: Americans want better prepared teachers in the classroom – and a vast majority think educators should be required to pass board certification and submit to licensure standards like doctors and lawyers. Those views come from a PDK/Gallup poll, released today. Seventy percent of respondents said new teachers should spend at least a year teaching under the guidance of a certified colleague. And 60 percent said the entrance requirements for teacher training programs need to be more rigorous. The results come as the Obama administration plans to resurrect an effort to regulate teacher prep programs. They also reflect public attitudes about whether the standardized testing regime ushered in by No Child Left Behind has improved education, said William Bushaw, who until recently served as executive director of PDK. Segun Eubanks, director of teacher quality at the National Education Association, says it’s clear all that testing hasn’t boosted student learning. So naturally, the focus is now swinging to improving the teaching profession. I have the story: http://politico.pro/1uEZjLi

– In another intriguing finding, 61 percent percent of the 1,001 adults surveyed opposed using student test scores in teacher evaluations. On a related note, researchers at The Brookings Institution are out with a study today that argues improving teacher observations is the key to upgrading evaluations. Observations are often biased by student ability and background, the authors say; they urge districts to adjust their observation scores accordingly. The study, published in Education Next: http://bit.ly/1wxBpUV

Anthony Cody notes a rift among allies. Mark NAISON wrote critically about PAR–Peer Assistance and Review.

Writing from his experience, Cody explains how PAR works.

The bottom line is this: If choosing how to be evaluated as a teacher, would you rather be evaluated by the rise or fall of test scores (VAM); by the principal, acting alone; or by a committee of peers and administrators whose first obligation is to help you?

Educators in Néw York are trying to make sense of the state’s evaluation system. The formula is supposed to consist of observations (60%); state scores (20%); and local assessments (20%). Yet the results don’t line up with common sense or common knowledge.

Some principals seem to be giving higher observation scores to teachers they want to protect because they believe they are valuable and don’t want to lose them

“In Scarsdale, regarded as one of the best school systems in the country, no teacher has been rated “highly effective” in classroom observations. It is the only district in the Lower Hudson Valley with that strict an evaluation. In Pleasantville, 99 percent of the teachers are rated as “highly effective” in the same category.”

Charlotte Danielson, whose rubric is the basis forest teacher evaluation systems, called these results “laughable.”

“Pleasantville schools Superintendent Mary Fox-Alter defended her district’s classroom observation scores, which use the Danielson model — saying the state’s “flawed” model had forced districts to scale or bump up the scores so “effective” teachers don’t end up with a rating of “developing.”

What is truly laughable is the effort to turn the art and craft of teaching into a scaled metric, like weighing apples at the supermarket. What is essentially a matter of human judgment, based on experience and wisdom, cannot be measured and graded. Its results will always be flawed, and the very act of measuring the unmeasurable will change teacher behavior to conform to the scale. If all we want is higher scores, this might be a good way to get them. If we want inspired teaching, it is not.

Paul Karrer, who teaches in Castroville, California, writes a scorching review of what is laughingly called “reform.”

He begins:

“Arne Duncan and his patron President Barack Obama have gotten themselves in a bit of an educational bind. Big news came out of the White House on Aug. 21 but a lot of America missed it. It seems a collision course of: 1. sunsetting of the year 2014 and the imbecilic impossible fatwa of No Child Left Behind (the obscenity of schools held accountable for testing without a morsel of input for poverty); and 2. a large push by teacher unions to dethrone he of the basketball — Sir Arne Duncan.”

So Duncan made his statement about testing “sucking the oxygen” out of teaching, a typical Duncanism in which he denounces the policies he promote and still enforces.

Says Karrer of Duncan’s fancy step:

“Is it a complete flip flop? No, it is a little greasy middle-of-the-road weaseling meant to gain favor from Obama’s once-upon-a-time education supporters and to patch the rebellious hemorrhaging of his pet bamboozle Race To The Top and its ugly stepsister Common Core. Ever since Obama initiated his slash and burn policy regarding public education with pro-privatization, the green light to pro-charter corporations, his relationship with publishing-testing companies, and his knee in the groin and knife in the backs of teachers with rigorous evaluations based on kids’ test scores, he’s been trusted about as much as a pedophile at a playground by those who once-upon-a-halo included him in their sacred prayers.”

Karrer says time is running out for the Age of Test and Punish. More and more people are speaking up and the public is catching on to the failure of test, test, test. The momentum is growing. Time is running out.