Archives for category: VAM (value-added modeling)

Bruce Lederman is suing the state of New York on behalf of his wife Sheri Lederman, a fourth grade teacher in the public schools of Great Neck, Néw York. The Ledermans contend that the state teacher evaluation system is irrational, and Bruce collected affidavits from leading scholars to support his claim, as well as laudatory statements from students, parents, and Sheri’s principal and superintendent.

Alexandra Milletta, a teacher educator and high school classmate of Sheri’s, attended the trial and reported her impressions on her blog.

She wrote:

“What I witnessed was a masterful take down of the we-need-objectivity rhetoric that is plaguing education. So I should begin by saying that I am hopeful, because it seems someone with the power to make a difference gets it. Judge McDonough gets that it’s all about the bell curve, and the bell curve is biased and subjective….

“As you may notice, we’ve come a long way from getting a 91 out of 100 on a test and knowing that was an A-. Testing today is obtuse and confusing by design. In New York State, we boil it down to a ranking from one to four. That’s right, there’s even jargon for “ones and twos” that is particularly heinous when you learn that politicians have interests in making more than 50% of students fall in those “failing” categories. Today the state released the test score results for students in grades 3-8 and their so-called “proficiency” is reported as below 40% achieving the passing levels. By design the public is meant to read this as miserable failure.

“The political narrative of public education failure extends next to the teachers, who must demonstrate student learning based on these faulty tests, even if they don’t teach the subjects tested, and even if they teach students who face hurdles and hardships that have a tremendous impact on their ability to do well on the tests. In Sheri’s case, her rating plunged from 13 out of 20 points to 1 out of 20 points on student growth measures. Yet her students perform exceedingly well on the exams; once you are a “four” you can’t go up to a “four plus” because you’ve hit the ceiling. In fact, one wrong answer could unreasonably mark you as a “three” and you would never know. Similarly, the teacher receives a student growth score that is also based on a comparison to other teachers. When it emerged in the hearing today that the model, also known as VAM, or value-added, pre-determined that 7% of the teachers would be rated “ineffective” Judge McDonough caught on to the injustice that lies at the heart of the bell curve logic: where you rank in the ratings is SUBJECTIVE…..

The State’s representative, Colleen Galligan, tried to defend the indefensible:

“The lame explanation from Colleen Galligan was that the model may not be perfect but the state tries to compare each student to similar students. The goal, she offered, is to find outliers in the teaching pool who consistently have a pattern of ineffectiveness, to either give them additional training or fire them. At this point Judge McDonough offered her a chance to explain the dramatic drop in Sheri’s score. “On its face it must mean students bombed the test (speaking as one who has bombed tests)” and this produced laughter in the courtroom. For who hasn’t bombed at least one test in their life? Who has not experienced that dread and fear of being labeled a failure? Then Judge McDonough asked rhetorically, “Did they learn nothing?” The only answer she could come up with, was that in this case Dr. Lederman’s students, although admittedly performing well compared to other students, did worse than 98% of students across the state in growth. At this point it was pretty clear to everyone present that this made absolutely no sense whatsoever.”

Alexandra believes and hopes that this trial may be the beginning of the end for VAM and other misuses of test scores to rank and rate teachers.

Peter Greene did not attend the trial, but he cut to the chase: “God Bless Sheri Lederman!” I would add to that “God Bless Bruce Lederman” for fighting for his wife and her professional reputation. Together, the Ledermans are fighting for all teachers.

Peter read Alexandra Miletta’s post, cited above. He writes:

“The New York teacher is in court this week, standing up for herself and for every teacher who suffers under New York’s cockamamie evaluation system. If she wins, there will be shockwaves felt all across America where teachers are evaluated based on VAM-soaked idiocy….

“Talking about the curve is the best way to help civilians understand why these teacher eval systems are giant heaps of baloney. If you’re old enough, you remember curves because they suck– get yourself in a class with the smart kids who all score 100% on a test and suddenly missed-one-question 95% is a C. Of course, younger civilians may not have such memories of the curve because over the past few decades most teachers have come to understand that curving is not a Best Practice.

“Evaluating teachers on the curve means that even if the VAM-sauce score actually meant something, the teacher evaluation itself will not mean jack. In a system in which every single teacher is above the bar in excellence, those teachers who are the least above the bar will be labeled failures.”

Maybe one thoughtful judge will put the VAMMERS in their place: out of the classroom.

Carol Burris went to Albany to attend the trial of Sheri Lederman’s case against the state of New York, which rated her “ineffective” based on her students’ growth scores. Many other educators attended the trial, which has national implications.

Sheri is an outstanding fourth grade teacher in a high-performing district. When she learned of her poor, computer-generated rating, she was devastated. But her husband Bruce, an attorney, determined to sue the state. He gathered affidavits from some of the mation’s leading experts on teacher evaluations, as well as students, teachers, and her principal.

At the trial, the judge recognized that grading teachers on a curve made no sense.

Burris reports:

“The exasperated New York Supreme Court judge, Roger McDonough, tried to get Assistant Attorney General Galligan to answer his questions. He was looking for clarity and instead got circuitous responses about bell curves, “outliers” and adjustments. Fourth-grade teacher Sheri Lederman’s VAM score of “ineffective” was on trial.

“The more Ms. Galligan tried to defend the bell curve of growth scores as science, the more the judge pushed back with common sense. It was clear that he did his homework. He understood that the New York State Education Department’s VAM system artificially set the percentage of “ineffective” teachers at 7 percent. That arbitrary decision clearly troubled him. “Doesn’t the bell curve make it subjective? There has to be failures,” he asked.

“The defender of the curve said that she did not like the “failure” word.

“The judge quipped, “Ineffectives, how about that?” Those in attendance laughed.

“Ms. Galligan preferred the term “outlier.” Those who got ineffective growth scores were “the outliers who are not doing a good job,” the attorney said. She seemed oblivious to the fourth-grade teacher who was sitting not 10 feet away from where she stood.

“Did her students learn nothing?” Justice McDonough asked. “How could it be that she went from 14 out of 20 points to 1 out of 20 points in one year?” He noted that the students’ scores were quite good and not that different from the year before.

“Back behind the bell curve Ms. Galligan ran. As she tried to explain once again, the judge said, “Therein lies the imprecise nature of this measure.”

Burris demonstrates the irrationality of the state’s measures. Teachers in some of the lowest-performing schools were rated “effective” or “highly effective,” while more teachers is some of the state’s best schools were rated “ineffective.” Crazy!

Burris writes:

“At its core, this story is a love story. It is the story of a teacher who loves her students, her profession and justice so much that she is willing to stand up and let the world know that she was “an outlier” with an “ineffective” score.

“It was love that compelled teachers, retired and active, driving from all corners of the state to be in that courtroom to listen on a hot summer’s day. It was love that compelled her principal to drive to Albany to be there. It was the deep and abiding love of a husband for his wife that compelled Bruce Lederman to spend countless hours preparing an extraordinary defense. And it is love that nourishes and sustains the good school, not avatar score predictions for performance on Common Core tests.”

Arthur Goldstein teaches English as a Second Language students at Frances Lewis High School in Queens, New York. He blogs as NYC Educator. In his letter, Goldstein refers to a meeting that Chancellor Farina had with a local superintendent, where she recognized that highly rated teachers were likely to get lower ratings in high-poverty schools. The blogger Perdido Street School wrote: “The dirty secret of education reform is that the problems in schools and districts with high poverty/high homelessness demographics are NOT caused by “bad teachers” – they’re caused by all the effects that poverty has on the psychological, emotional, physical and social development of the children in those schools and districts.”

Arthur Goldstein writes:

Dear Chancellor Fariña:

First of all, I applaud you for acknowledging that a highly-effective rated teacher entering a troubled school may suffer a reduced rating as a result of changing schools. I very much appreciate that you’ve taken a personal interest in this teacher and plan to attach an asterisk and follow her ratings. It’s inspirational not only to me, but also to teachers nationwide, that the leader of the largest school district in the country would acknowledge that a school’s population is a major factor in teacher ratings.

This, in fact, has been a major objection many of us, including experts like Diane Ravitch and Carol Burris, have had toward value-added evaluation programs. In fact, the American Statistical Association has determined that teachers impact test scores by a factor of 1-14%. They have also determined that rating teachers by such scores may have detrimental effects on education.

I am struck by the implications of your statement. If it’s possible that a highly-rated teacher may suffer from moving to a school with low test scores, isn’t it just as likely that a poorly-rated teacher would benefit from being moved from a low-rated school to a more highly-rated one? And if, as you say, the teachers are using the same assessments in either locale, doesn’t that indicate that the test scores are determined more by students themselves as opposed to teachers?

For example, I teach beginning ESL students. Teaching these kids is one of the very best things I’ve ever done, but I now consider it a very risky business. Kids who don’t speak English tend not to achieve high scores on standardized tests. I’m sure you also know that acquiring English takes a few years, varies wildly by individual, and that it can take 5-7 years to acquire academic English. The new NYSESLAT test seems to focus on academic English rather than language acquisition. Still, it would be irresponsible of me to neglect offering basic conversation and survival skills. (In fact, NY state now requires that we offer less standalone ESL., which is neither helpful to my students nor supported by research.)

Special education children also have specific needs and disabilities that can inhibit their ability to do well on tests. It doesn’t take an expert to determine that teachers in schools with high concentrations of students with disabilities already are more likely to incur adverse ratings. Who is going to want to teach in these schools? Who will want to teach special education or ESL?

Attaching high stakes to test scores places undue pressure on high-needs kids to pass tests for which they are unsuited. For years I’ve been hearing about differentiation in instruction. I fail to see how this approach can be effectively utilized when there is no differentiation whatsoever in assessment. It’s as though we’re determined to punish both the highest needs children and their teachers.

Since the advent of high-stakes evaluations, the morale of teachers I know and represent has taken a nose dive. Teachers, regardless of ratings, are constantly asking me about their ratings, and live in fear of them, as though the Sword of Damocles were balanced over their heads. Though the Danielson rubric is heralded as objective, in practice it’s very much in the eye of the beholder. As if that were not enough, ratings are frequently altered by test score ratings. Diane Ravitch characterizes them as junk science. (I concur, and having music teachers rated by the English Regents scores of their students pushes it into the realm of the ridiculous.)

Personally, I found the older evaluation reports to be much more thorough and helpful. Supervisors used to be able to give detailed reports of what they saw, and specific suggestions on what could be improved. ThoughI can’t speak for everyone in this, I found them easier to read than the checklists we currently receive. Just like our kids, we are not widgets. We are all different, and are good or not so good on our own merits.

Of course no one wants bad teachers in front of children. The current system, though, seems to focus on student test scores rather than teacher quality. It seems to minimize teacher voice in favor of some idealized classroom that may or may not exist.

It’s a fact that test scores are directly correlated with family income and level of special needs. There is no reliable evidence that test scores are indicative of teacher quality or lack thereof. Teachers are the second-best role models for children. It’s quite difficult for us to show children that life is a thing to be treasured when we have virtual guns placed to our heads demanding higher test scores or else. Just like our kids, we are more than test scores.

On behalf of children and teachers all over New York State, I ask that you join us in demanding a research and practice-based system of evaluating not only teachers, but our students as well.

Sincerely,

Arthur Goldstein, ESL teacher, UFT chapter leader
Francis Lewis High School

From a reader:

“Florida VAM formula … from the DOE website. This is a terrible joke.

y_i=μ+∑_(g=1)^M▒〖δ_g x_g 〗+∑_(j=1)^K▒〖β_j x_j+θ_(k)i+ω_(mk)i+ε_i; 〗

“VAM is a SCAM and my children will be no part of it.”

Recently the website Five Thirty Eight published a post declaring that VAM works. Only economists who never set foot in a school since their own high school days could reach this conclusion.

This reader disagrees with the economists:

This was my comment, as a lowly parent & taxpayer, at the 538 article comment thread:

As a taxpayer and parent I’d like to know what this article has to do with economics. I can find nothing in the article, nor in the comments thread, which speaks to (much less ‘proves’) that VAM shows any results which save the taxpayer money. As a member of a community I am well aware of the time our teachers have been spending for the last two years on the voluminous paperwork required to implement VAM. Although they seem to be doing it without extra pay, this is worrisome. These people already spend too many hours on a medium-salary job, and with the extra stress, that will push more of them out of a difficult profession.

What’s worse at least in my area (NJ), VAM seems to be a 2-pronged affair:

there’s the SLO/ Danielson or Marzano business that has them adding short autumn and EOY assessments in every subject to measure progress — which sounds like debunked & easily-gamed MBO junk I had to do in the ’80’s private sector–

and then we have the students’ PARCC scores which will soon become a weighted part of the teachers’ annual evaluation.

The SLO business adds a couple of hrs’ testing in every subject (so 10 hrs of class time per year)– minimal yet to my mind from experience a waste of time.

The latter– PARCC CCSS-aligned tests, taken on computer, disrupted the schedule horribly this year. Just to administer them required weeks during March & May to shuffle students by group in & out of gym/ library/ computer lab, where available laptops were corralled. That meant those areas were lost to normal physed, library research projects, curriculum-associated computer-lab work, for 3 – 4 wks of the 12-wk spring semester.

Much class time was spent teaching primary students how to keyboard, drag-&-drop, scroll, not hit buttons that would freeze the screen. And much more class time was spent drillimg kids on the PARCC Q&A format.

Although the PARCC tests are administered solely to collect data on school & teacher performance (meaning by the way older students recognize they have no ‘skin in the game’)– because they are or soon will be hi-stakes for school & staff, much time & attention is paid.

Did I mention my district has long been one of the highest-performing in the state? Here in NJ rich districts like ours send a large chunk of our RE taxes to poor districts; we pay 96% of the school budget ourselves. And now we must sacrifice many weeks of our precious, highly-paid-for curriculum-learning time to… VAM???

OK now I’m mad. I can read right on this board that there’s absolutely no proof that any of this supposedly economically-driven activity has anything to do with lowering my property taxes. In fact, what I can glean by reading the national news, its primary effect to date has been to drive veteran teachers into early retirement, to be replaced by substitutes, TFAs, & newbies (which is what I see is happening in Newark, where much of my RE taxes go, which is why mine are cripplingly high; as far as I can determine costs are higher than ever in fact Zuckerberg’s $100million went to consultants and state administrators).

Audrey Amrein-Beardsley writes in her blog VAMboozled that the VAM-loving economists are giving each other high-fives for the triumph of VAM. Almost every state has adopted some version of it. Success!

 

What they don’t offer is any evidence that VAM has improved education.

 

Where are the districts or the states that have identified and fired the ineffective teachers and seen a vast improvement in their schools?

 

Why do the VAM ratings continue to bounce around? Why are teachers known to be successful in their school community getting low VAM ratings? Why are teachers often rated based on the scores of students they never taught?

 

Does reality matter?

Politico.com reports today that the General Accounting Office wants the U.S. Departmemt of Education to exercise greater oversight over teacher education programs. The question is how quality will be judged? Will it be the pass rates on Pearson’s EdTPA? Or the VAM ratings based on student test scores after graduation? If the former, expect to see a sharp decline in the proportion of African-American and Latino teachers? If the latter, expect to see teachers avoiding special education and schools in poor districts?

Incentives have unintended consequences.

As long as they are beefing up oversight at ED, why don’t they close down some of the predatory for-profit colleges that sell worthless diplomas and saddle young people with debt?

If the U. S. Department of Education had the capacity to oversee any sector,

Rhode Island Governor Gina Raimondo is a former venture capital entrepreneur. As state treasurer, she redirected the state’s pension funds. Her husband Andy Moffitt is a co-founder of the Global Education Practice at McKinsey. He is active with the anti-union, anti-teacher Stand for Children. He was a member of Teach for America. Moffitt co-wrote (with Paul Kihn and Michael Barber) “Deliverology 101: A Field Guide for Educational Leaders.”

The blog site “RIFuture” wrote of McKinsey:

“In terms of corporate education reform, one prominent McKinsey-watcher and follow-the-money researcher puts the firm in a class by itself:

“They have been the leaders in crafting the dominant narrative of an education crisis for decades, and now deeply entrenched in education reform policies, they are reaping the financial and political benefits of marketing solutions to the problems they manufactured in the first place.”

Governor Raimondo recently selected Deputy Commissioner of Education Ken Wagner as the new State Superintendent in Rhode Island. In Néw York, he was known as a strong supporter of high-stakes testing, VAM, and corporate reforms.

Sheila Resseger, a teacher in Rhode Island for many years, was unhappy with Raimondo’s choice. She wrote, in response to a post about Néw York’s Common Core curriculum called EngageNY:

“Here was my comment to the post that Diane referenced. I am going to make it my mission to inform Rhode Islanders about the total disdain that Ken Wagner has for authentic teaching and learning. According to the RI Dept of Ed and Gov. Raimondo, he “developed” EngageNY. By his own admission he is opposed to Opt Out and for data collection. These are the trifecta of evil in my book: Common Core/Pear$on testing/data mining.

“I find this so profoundly disturbing that I can hardly see straight to type this comment. I live in RI. As you may know, our Governor, Gina Raimondo, recently nominated NY State Deputy Commissioner of Ed Dr. Ken Wagner to be our new Commissioner of Education (replacing Broad-trained Deborah Gist). This past Monday night the RI Board of Education and Council on Elementary and Secondary Education met to decide whether or not to confirm Dr. Wagner. I was the only one to speak against his confirmation. Dr. Wagner was credited with developing EngageNY, and seemed to be delighted that it has been downloaded for free 20 million times. He also declared that the Common Core does not script lessons, but actually frees up teachers to teach creatively. Another egregious comment of his was that we don’t have to be concerned with Piaget’s developmental stages–that theory is passé. Now we know that children can do so much more than we had expected of them before. Yes, every first grader is delighted to learn about the Code of Hammurabi.

“Here is my post in RIFuture.org, published before the meeting. http://www.rifuture.org/will-ken-wagners-past-in-new-york-shape-his-future-in-rhode-island.html”

Laurie Gabriel, a teacher with nearly three decades experience, decided that she had to do something to fight back against the absurd attacks on teachers.

 

The first thing she did was to create a documentary to explore the critical issues of the day. It is called “Heal Our Schools,” and it offers practical advice that most teachers would vigorously agree with. In her video, she interviews teachers, students, and a few outsiders (like me). The people she spoke to talked about what matters most in teaching and learning, which she would say is to encourage students to find their passions and pursue them. Her first recommendation, by the way, is to reduce class size so children can get individual attention when they need it.

 

The high point of the film, in my estimation, was when she spoke to some vocal critics of teachers. She invited them to teach a list of vocabulary words to ten students, and they accepted her offer. The scenes were priceless. The students were restless; one put his head on the desk. Announcements on the public speaker interrupted the lessons. When one of the “teachers” reprimanded a student and told him that when he was in the Army, he would have gotten 50 push-ups for his behavior, another student piped up and said, “We’re not in the Army.” After their students took their tests, Laurie gave them feedback about their performance. They were less enthusiastic about grading teachers by their students’ test scores and even seemed to be more respectful of the skill that it takes to teach middle schoolers.

 

The second thing she did was to take her documentary on the road, showing it to interested audiences. Her current schedule starts tonight in Wyandotte, Michigan. You can see her other stops listed below. If you live in one of these cities or towns, please show up and bring some friends.

 

July 21 WYANDOTTE MI (Detroit area)- 7:00 at Biddle Hall, 3239 Biddle Ave.
July 22 CLEVELAND – 7:00 at the West Shore Unitarian Universalist Church
July 23 PHILADELPHIA – 7:00 at the Ethical Humanist Society, 1906 Rittenhouse Square
July 24 WASHINGTON DC – 8:00 at the Holiday Inn Washington Capitol, 550 C Street SW
July 26 – JERSEY CITY – 7:00 pm at the Jersey City Union Building, 1600 W. Kennedy
July 28 NEW YORK CITY – 2:30 pm at the Actors Theatre Workshop, 145 W. 28th Street, 3rd floor
July 29 RAYNHAM MASS. 6:00 pm at the Massachusetts Teachers Regional Office, 656 Orchard Street 3rd floor
July 30 PORTSMOUTH NH, 7:00 pm at the Women’s City Club, 375 Middle Street
August 3 – GRAND BLANC, MI – 6:00 at the Grand Blanc Mcfarlen Public Library.
August 9 DENVER – 1:00 pm at the Highlands Ranch Public Library, 9292 Ridgeline Blvd in Highlands Ranch

 

If you don’t live in one of those locales and want to see “Heal Our Schools,” contact Laurie at aspenquartet@hotmail.com

 

Perhaps you could arrange a showing in your community.

This article about the lack of evidence for VAM is open access for one week only.

It was written by VAM critic Audrey Amrein-Beardsley and Jessica Holloway-Libell.