Archives for category: Teacher Evaluations

Fred Klonsky’s blog carries a post by retired educator Sandra Deines about a fateful decision in Illinois:

“Starting this fall Pearson will be in the business of deciding who becomes a teacher in the state of Illinois.

“The Illinois State Board of Education has adopted a rule that designates Pearson’s “edTPA” as the means by which student teachers will be evaluated and granted certification.

“As the fall semester begins, all student teachers in the state will be required to pay an extra $300 (on top of the tuition they are already paying) and arrange for videotaping so that they can submit a lengthy narrative that covers the planning, execution and evaluation of a series of lessons with one of their classes as well as a ten-minute video of themselves carrying out their lesson with a class.

“Student teachers are required to get parent permission for their children to be video-taped.

“Pearson owns the video.

“Once submitted to Pearson, an “evaluator” will apply rubrics and 2-3 hours of their time to decide whether or not the student teacher “passes” and can be licensed to teach by the State of Illinois.

“That’s right—no longer will the evaluations of cooperating teachers, university field instructors and education professors determine the success of a student teacher.”

To learn about how to resist the Pearson takeover of teacher certification in Illinois, read the test of the post.

Carol Burris recently became executive director of the Network for Public Education Fund. The NPE Fund is the nonprofit, nonpolitical, nonpartisan wing of NPE, as opposed to NPE’s c(4), which endorses political candidates and is led by Robin Hiller of Tucson.

She plans to issue regular reports on important education issues. A prolific and well-informed writer, her perspective will help to inform and hopefully shape the national debate about education.

In this post, she explains the causes of the national teacher shortage. As she writes, the New York Times attributed it to an improving economy, which opened up more attractive jobs than teaching (hmmm, given the collapse of the stock market, maybe the shortage will end soon?).

Burris says the economy may have something to do with the shortage, but other factors were also important:

Earlier this year, NPR also reported on the national teacher shortage. Correspondent Eric Westervelt’s identification of the cause went beyond the usual suspect—the economy. Noting the dramatic drop in enrollment in teacher education programs (a 74% drop in less than 10 years in California), he astutely attributed at least part of the problem to the way corporate reforms have impacted the profession.

Westervelt reported that the Common Core and its battles; high-stakes testing, the erosion of tenure, and the evaluation of teachers by test scores, have all contributed to the crisis.

This comes as no surprise to those inside the profession.

David Gamberg is the superintendent of the Greenport and Southold districts on Long Island’s east end. He has long worried that the politically hostile environment for teachers is contributing to the shortage we are seeing today. “I suspect that a range of issues conspire to exacerbate the problem. Certainly the ongoing, nationwide attack on teachers and unions is near or at the very top of the list of factors driving people away.”

What Gamberg suspects has evidence. There are frequent stories about public school teachers who are leaving the profession or taking early retirement because of the toll of working in a ‘test and punish’ environment. A November NEA survey reported that nearly 50% of all teachers are considering leaving due to standardized testing. Of equal concern is how frequently educators are cautioning young adults about entering the profession.

Renowned author and teacher of literacy, Nancie Attwell, recently won the first annual $1 million Global Teacher Prize awarded by the Varkey Foundation. When she was asked by CNN whether she would advise others to become a public school teacher, her response was she would not. She said she would tell them to find a job in the private sector, or in an independent school instead. She spoke about how constricting both the Common Core and testing have made the profession. “If you’re a creative, smart young person, I don’t think this is the time to go into teaching unless an independent school would suit you.” she said.

EdWeek reported on the story, which was followed by a poll. By nearly a 5 to 1 margin, respondents said that they would not recommend teaching as a profession. Considering that EdWeek readers are by and large educational professionals, that response, combined with the NEA data, is a clear indicator of the stress felt within the profession from outside reforms.

If we are to turn this trend around, we need to act now to not only stop the attacks on teachers and tenure, but to stop evaluation systems designed to fire teachers based on metrics that no one understands. And we cannot forget that pay and working conditions matter. It should also come as no surprise that in states that pay teachers relatively well like New York State, the shortage does not yet exist. Even so, enrollment in teacher preparation programs in the Empire State dropped 22% in two years time. Many factors are contributing to the decline.

It is time for policymakers to step back and chart a different course. It makes no sense to cling to failed reforms. As school begins, students across the country are paying a hefty price.

One of the affidavits at the trial of the Lederman v. King case was filed by psychologist Brad Lindell.

His full affidavit is included in this post, which contains all the affidavits.

He sent the following note to me to explain his view of VAM in layman’s terms:

I am Dr. Brad Lindell, one of the affiants in the Sheri Lederman case who was present at the oral arguments on Wednesday. It was truly something to observe. You got the feeling that good was was going to come from the great work of Sheri and Bruce Lederman and from the experts’ opinions in so far as changing this broken VAM system. You got the sense that the judge was listening to the science about VAM and not just to the political rhetoric.

Just want to fill you in on something that was presented in my affidavit modified to give a clear and understandable example of the effects of poor reliability on a full-scale WISC intelligence test. If the same test-retest reliability from the teacher assigned yearly VAM scores (.40) was applied to the WISC full-scale to determine the 90% confidence interval, the range would be ridiculously large.

Examples

If a student scored a full-scale IQ of 100 (average) then the 90% confidence interval would be an 81 to 119. This indicates that there would be a wide range where the scores from repeated administrations of the WISC would be expected to fall for this student. One could not have confidence in the validity of a intelligence test with low reliability. Without adequate reliability, there can not be validity. This same holds true for VAM scores, whose reliabilities have been found to be notorious low.

The reliability of the WISC is generally in the .80 to .90 range. The 90% confidence intervals are generally in the +\- 6 range. So this same person with a 100 full-scale IQ would have a 90% confidence range of 94-106. Quite a smaller range.

This is why reliability is so important, which has repeatedly been shown to be low like .2 to .4 for year-to-year VAM scores. This is also why teachers year to year VAM score vary so considerably, like in the case of Sheri Lederman. Without reliability there cannot be adequate validity.

Mike Petrilli of the Thomas B. Fordham Foundation/Institute has been a strong supporter of school choice and the Common Core. On the whole, he and TBF have applauded Arne Duncan’s move to promote charter schools, to ignore the voucher proliferation, and to push Common Core on the states (as if they were “state-led,” which they were not).

However, Petrilli now has had a change of mind. (For the record, I support those who are willing to rethink their views and change their minds.) He now recognizes that Arne overreached and caused a counter-reaction. The most atrocious action by Duncan was to force test-based teacher evaluation on the states, with no evidence that it would improve education. It was a disaster. It hasn’t worked anywhere, and it has increased teaching to the test and teacher demoralization. If you are looking for the cause of the widespread teacher shortage, look to the policies of the U.S. Department of Education since 2009.

Petrilli writes, with humility, that he was wrong.

It’s not just that the Department of Education usurped power from Congress and the states; it’s that they used that power to push bad policy. Nobody today can creditably argue that mandating statewide teacher evaluations as a condition of ESEA flexibility was a good idea. Nobody can say that the teacher evaluation efforts are going well. This was an unforced error of enormous magnitude—one that has sparked a significant backlash to accountability policies writ large and also destroyed whatever credibility the feds may have had….

So yes, both the Senate and House versions of ESEA reauthorization are “looser” than No Child Left Behind, or than the Fordham proposal from 2011. If this renewal processes gets across the finish line (and I think it will), the federal government will have much less power than it does today. Folks like Chad who don’t like that will only have Arne Duncan to blame.

Everyone understands that the key fact about Néw York’s test scores is that they will be used to measure the “effectiveness” of teachers. The progress of children has been small over three years, and the scores align closely with demography, language, disability, and family income. Ho-hum.

Mercedes Schneider reminds us of basic facts:

“Under no conditions is it a valid use of student test scores to evaluate teachers or schools.

“The students are the test takers; these tests purportedly measure their achievement. There is no way to account for all of the possible variables that would enable the New York State Education Department (NYSED) to accurately evaluate teachers and schools using student test scores.”

A number of teachers from the Bad Ass Teachers Association drove to Albany to witness the trial of Néw York’s teacher evaluation system. Here are excerpts from some of their reports.

BAT 1:

“In responding to the Lederman’s suit, the state representatives Ira Schwartz, Assistant Commissioner of Accountability and (?) Sherman, a “Quality Control” official did not provide affidavits from independent experts, rather they asserted that the Lederman’s misunderstood the meaning of “growth”, providing language from promotional brochures.

“But the response also conceded that the policy was derived in pursuit of federal Race to the Top funding (another case of “Thanks Obama?”)
The judge teased this out of the state’s lawyer by asking so the students can perform well and the teacher be deemed ineffective? The Lawyer eventually answered “yes!” but tried to remind the court that the formula is only used as part of the overall evaluation. This caused the judge to ask aloud why “discordant” and “inappropriate” results would be used in any percentage.”

BAT 2:

“In comparison to Lederman’s pointed and constructed argument, the assistant attorney general did a minimal response to argument, defending the definition and use of the growth model. The judge asked over and over how a teacher could go from a 14 one year to a 1 the following year. No answer was really given except for a poor attempt to explain the model’s comparison to other students.

“Quote of the day – from Lederman – went something like this: Are we living in a science fiction world where Hal the Computer gets to make decisions and there is no opportunity for human input or appeal?…

“I am in awe of the Lederman’s, true heroes for all of the downtrodden teachers being judged by a flawed and unfair system of measurement. If they win, all teachers and all students win.”

BAT 3:

“Another thing that stood out was just how deeply flawed the system is. When a teacher like Sheri goes from 14 points to 1, yet her students are doing very well and meeting proficiency, it’s easy to see that something is deeply wrong. I believe everyone in the courtroom saw that today. The argument that it’s just a portion of an overall score doesn’t matter to me. If any part of it is flawed, the whole thing should not be used. Let’s hope the judge agrees.

“Lastly what stood out was how much Bruce shined, and the state faltered. Prepared and eloquent, Bruce laid out the arguments point by point and handled challenging questions with thorough and thoughtful explanations. The same could not be said about the state’s representation and it was wonderfully obvious.”

BAT 4:

“It was exciting for me to witness this hearing, and I feel that the outcome of this case could be very historic in our fight to save public education.”

BAT 5:

“The state continued to argue the rating computer system was valid. NYSED admitted that you can have a teacher whose kids are successful yet the teacher will still get a low rating. Overall, they seemed okay with that reality. The state explained the evaluation system will let them get rid of ‘outliers’.

“After the proceedings I was moved to tears when The Lederman’s shared their motivation. Mrs. Lederman’s evaluation had her distraught, she was ready to quit and leave the profession that she loves. After many late night discussions, she and her husband decided, to challenge this unjust system. They certainly know they have a world of educators and parents on their side.”

BAT 6:

“* It was encouraging to have the public see for themselves that even the state could not explain the impact of test scores on teachers.

* The common sense scenarios as presented by the judge made it clear to everyone in the courtroom just how ridiculous VAM is.”

BAT 7:

“I am biased, for sure, but I have to say Bruce and Sheri Lederman have done an amazing job laying out the faults of this system. They have lined up the best in the field to validate their claims. My overall feeling was, that as gruff as this judge seemed to be, he got it. He understood that this system is not transparent, is not valid, and should not be used to judge teachers. We hope that his findings, which will be rendered in about 6 weeks, will state just that.”

Bruce Lederman is suing the state of New York on behalf of his wife Sheri Lederman, a fourth grade teacher in the public schools of Great Neck, Néw York. The Ledermans contend that the state teacher evaluation system is irrational, and Bruce collected affidavits from leading scholars to support his claim, as well as laudatory statements from students, parents, and Sheri’s principal and superintendent.

Alexandra Milletta, a teacher educator and high school classmate of Sheri’s, attended the trial and reported her impressions on her blog.

She wrote:

“What I witnessed was a masterful take down of the we-need-objectivity rhetoric that is plaguing education. So I should begin by saying that I am hopeful, because it seems someone with the power to make a difference gets it. Judge McDonough gets that it’s all about the bell curve, and the bell curve is biased and subjective….

“As you may notice, we’ve come a long way from getting a 91 out of 100 on a test and knowing that was an A-. Testing today is obtuse and confusing by design. In New York State, we boil it down to a ranking from one to four. That’s right, there’s even jargon for “ones and twos” that is particularly heinous when you learn that politicians have interests in making more than 50% of students fall in those “failing” categories. Today the state released the test score results for students in grades 3-8 and their so-called “proficiency” is reported as below 40% achieving the passing levels. By design the public is meant to read this as miserable failure.

“The political narrative of public education failure extends next to the teachers, who must demonstrate student learning based on these faulty tests, even if they don’t teach the subjects tested, and even if they teach students who face hurdles and hardships that have a tremendous impact on their ability to do well on the tests. In Sheri’s case, her rating plunged from 13 out of 20 points to 1 out of 20 points on student growth measures. Yet her students perform exceedingly well on the exams; once you are a “four” you can’t go up to a “four plus” because you’ve hit the ceiling. In fact, one wrong answer could unreasonably mark you as a “three” and you would never know. Similarly, the teacher receives a student growth score that is also based on a comparison to other teachers. When it emerged in the hearing today that the model, also known as VAM, or value-added, pre-determined that 7% of the teachers would be rated “ineffective” Judge McDonough caught on to the injustice that lies at the heart of the bell curve logic: where you rank in the ratings is SUBJECTIVE…..

The State’s representative, Colleen Galligan, tried to defend the indefensible:

“The lame explanation from Colleen Galligan was that the model may not be perfect but the state tries to compare each student to similar students. The goal, she offered, is to find outliers in the teaching pool who consistently have a pattern of ineffectiveness, to either give them additional training or fire them. At this point Judge McDonough offered her a chance to explain the dramatic drop in Sheri’s score. “On its face it must mean students bombed the test (speaking as one who has bombed tests)” and this produced laughter in the courtroom. For who hasn’t bombed at least one test in their life? Who has not experienced that dread and fear of being labeled a failure? Then Judge McDonough asked rhetorically, “Did they learn nothing?” The only answer she could come up with, was that in this case Dr. Lederman’s students, although admittedly performing well compared to other students, did worse than 98% of students across the state in growth. At this point it was pretty clear to everyone present that this made absolutely no sense whatsoever.”

Alexandra believes and hopes that this trial may be the beginning of the end for VAM and other misuses of test scores to rank and rate teachers.

Peter Greene did not attend the trial, but he cut to the chase: “God Bless Sheri Lederman!” I would add to that “God Bless Bruce Lederman” for fighting for his wife and her professional reputation. Together, the Ledermans are fighting for all teachers.

Peter read Alexandra Miletta’s post, cited above. He writes:

“The New York teacher is in court this week, standing up for herself and for every teacher who suffers under New York’s cockamamie evaluation system. If she wins, there will be shockwaves felt all across America where teachers are evaluated based on VAM-soaked idiocy….

“Talking about the curve is the best way to help civilians understand why these teacher eval systems are giant heaps of baloney. If you’re old enough, you remember curves because they suck– get yourself in a class with the smart kids who all score 100% on a test and suddenly missed-one-question 95% is a C. Of course, younger civilians may not have such memories of the curve because over the past few decades most teachers have come to understand that curving is not a Best Practice.

“Evaluating teachers on the curve means that even if the VAM-sauce score actually meant something, the teacher evaluation itself will not mean jack. In a system in which every single teacher is above the bar in excellence, those teachers who are the least above the bar will be labeled failures.”

Maybe one thoughtful judge will put the VAMMERS in their place: out of the classroom.

Carol Burris went to Albany to attend the trial of Sheri Lederman’s case against the state of New York, which rated her “ineffective” based on her students’ growth scores. Many other educators attended the trial, which has national implications.

Sheri is an outstanding fourth grade teacher in a high-performing district. When she learned of her poor, computer-generated rating, she was devastated. But her husband Bruce, an attorney, determined to sue the state. He gathered affidavits from some of the mation’s leading experts on teacher evaluations, as well as students, teachers, and her principal.

At the trial, the judge recognized that grading teachers on a curve made no sense.

Burris reports:

“The exasperated New York Supreme Court judge, Roger McDonough, tried to get Assistant Attorney General Galligan to answer his questions. He was looking for clarity and instead got circuitous responses about bell curves, “outliers” and adjustments. Fourth-grade teacher Sheri Lederman’s VAM score of “ineffective” was on trial.

“The more Ms. Galligan tried to defend the bell curve of growth scores as science, the more the judge pushed back with common sense. It was clear that he did his homework. He understood that the New York State Education Department’s VAM system artificially set the percentage of “ineffective” teachers at 7 percent. That arbitrary decision clearly troubled him. “Doesn’t the bell curve make it subjective? There has to be failures,” he asked.

“The defender of the curve said that she did not like the “failure” word.

“The judge quipped, “Ineffectives, how about that?” Those in attendance laughed.

“Ms. Galligan preferred the term “outlier.” Those who got ineffective growth scores were “the outliers who are not doing a good job,” the attorney said. She seemed oblivious to the fourth-grade teacher who was sitting not 10 feet away from where she stood.

“Did her students learn nothing?” Justice McDonough asked. “How could it be that she went from 14 out of 20 points to 1 out of 20 points in one year?” He noted that the students’ scores were quite good and not that different from the year before.

“Back behind the bell curve Ms. Galligan ran. As she tried to explain once again, the judge said, “Therein lies the imprecise nature of this measure.”

Burris demonstrates the irrationality of the state’s measures. Teachers in some of the lowest-performing schools were rated “effective” or “highly effective,” while more teachers is some of the state’s best schools were rated “ineffective.” Crazy!

Burris writes:

“At its core, this story is a love story. It is the story of a teacher who loves her students, her profession and justice so much that she is willing to stand up and let the world know that she was “an outlier” with an “ineffective” score.

“It was love that compelled teachers, retired and active, driving from all corners of the state to be in that courtroom to listen on a hot summer’s day. It was love that compelled her principal to drive to Albany to be there. It was the deep and abiding love of a husband for his wife that compelled Bruce Lederman to spend countless hours preparing an extraordinary defense. And it is love that nourishes and sustains the good school, not avatar score predictions for performance on Common Core tests.”

The Lederman challenge to Néw York’s teacher evaluation system will be heard tomorrow morning at 10 am in the court of Judge McDonough at 16 Eagle Street in Albany. If you are within shouting distance, show up to give Sheri moral support. But mind your manners and respect the decorum of the courtroom.

If you are interested in the subject of teacher evaluation, here is a treat for you. This file contains the affidavits of the experts in the Lederman v. King case, which will be heard in New York Supreme Court in Albany on August 12 at 10 a.m. (If you have trouble with that link, try this one.)

It also contains statements from Sheri Lederman’s superintendent in Great Neck, New York, her principal, her former students, and parents, all testifying to her effectiveness as a teacher. It also includes an affidavit by an economist at AIR attempting to explain New York’s method of calculating teacher effectiveness, defending Sheri Lederman’s rating as ineffective.

Just reading all these affidavits should be enough to earn course credits at any college or university.

If Sheri Lederman should win, her victory will have statewide impact and even national impact.

If she should lose, it is the triumph of an incoherent and punitive status quo.