Archives for category: VAM (value-added modeling)

In an interview published in The Hechinger Report, Randi Weingarten expresses her belief that Hillary Clinton will change course from the Obama education policies. She expects that a President Clinton would select a new Secretary of Education, one who shares her expressed belief in strengthening public schools and supporting teachers.

Emmanuel Felton, who conducted the interview, writes:

While teachers unions have long been a key pillar in Democratic Party, they’ve been on the outs with President Barack Obama’s education department. The administration doubled down on Republican President George W. Bush’s educational agenda of holding schools accountable for students’ test scores. Under the administration’s $3 billion School Improvement Grant program, for example, struggling schools had options to implement new accountability systems for teachers, remove staff, be closed or converted into charter schools, the vast majority of which employ non-unionized staff.

These policies devastated some local teachers unions, including Philadelphia’s, which lost 10,000 members during the Obama and Bush administrations. Weingarten expects Clinton to totally upend this agenda, and hopes she won’t reappoint Education Secretary John King, who was just confirmed by the senate in March.

From the day he was elected, President Obama decided to maintain the punitive policies of George W. Bush’s No Child Left Behind and made standardized testing even more consequential. He and his Secretary of Education Arne Duncan pressed for higher standards, tougher accountability, and more choices, especially charter schools. They used Race to the Top to promote the evaluation of teachers by their students’ test scores, a policy that cost hundreds of millions, perhaps billions of dollars, with nothing to show for it.

Let’s all hope that Hillary Clinton, if elected, will recognize the damage done by the Bush-Obama education agenda and push the “reset” button for a federal policy that helps children, educators, and public schools.

Our blog poet has not commented lately. Poet, we miss you! Come back!

“The night they drove Statricksy down” (parody of “The night they drove Old Dixie down)

Thomas Kane is my name and I drove on the VAMville train

‘Til Audrey Beardsley came and tore up the tracks again.

After the ASA* paper knife , we were hungry, just barely alive.**

By twenty-fourteen, Rich man had fell.
It’s a time I remember, oh so well.

(*American Statistical Association, **only had $ 45 million from the “Rich man”, Bill Gates)

The night they drove statricksy down

And all the bells were ringing,

The night they drove statricksy down

And all the people were singing

They went, “Na,na,na.na,

Na na na na na na na na na.”

Back with my colleague, Raj Chet-ty, when one day he called to me,
“Thomas, quick, come see, there goes the Gatesly Billee!”
Now I don’t mind I’m choppin’ stats, and I don’t care if I’m paid by the brats
You take what you need and leave the rest,
But they should never have taken the VAMmy best.

The night they drove statricksy down
And all the bells were ringing,
The night they drove statricksy down
And all the people were singing
They went, “Na,na,na.na,
Na na na na na na na na na.”

Like my “Father”*** before me, I will work the VAM
And like my colleague before me, I took a junk-stat stand.
He was just 34, proud and brave,
but the ASA put him in his grave.
I swear by the mud below my feet
You can’t raise a Kane back up when he’s in defeat

(***William Sanders, Father of VAM)

The night they drove statricksy down
And all the bells were ringing,
The night they drove statricksy down
And all the people were singing
They went, “Na,na,na.na,
Na na na na na na na na na.”

The night they drove statricksy down
And all the bells were ringing,
The night they drove statricksy down
And all the people were singing
They went, “Na,na,na.na,
Na na na na na na na na na.”

Harvard Magazine published a noteworthy article about the failure of pay-for-performance plans in hospitals, written by Marina N. Bolotnikova. It is worth your time to read. It is devastating.

The logic of pay-for-performance systems is simple enough: pay doctors and hospitals based on how well their patients are doing, rather than on the number of medical services they provide. The payment structure was designed to fix a central problem in American healthcare. The United States spends far more per person on healthcare than any other country, yet has the poorest health outcomes in the advanced world. Pay-for-performance, also known as value-based purchasing, was meant to encourage doctors to optimize the welfare of patients while discouraging spending on unnecessary care.

There was never any evidence that pay-for-performance works, said Li professor of international health Ashish Jha, who investigates the effectiveness of economic incentives in the healthcare market. A recent paper published this April in the BMJ by Jha finds that the federal pay-for-performance program under Medicare, the Hospital Value-Based Purchasing (HVBP) program, hasn’t had any impact on mortality rates. Nor do HVBP-participating programs show any statistically significant advances over the small number of hospitals that don’t participate in the program. No hospitals, not even those with the worst mortality rates before the program was implemented, showed an improvement that could be attributed to pay-for-performance. “We looked across all different conditions and couldn’t find a single one where it seemed to have a meaningful effect,” Jha explained.

In the early 2000s, the Bush administration ran a pilot version of pay-for-performance at about 200 hospitals that agreed to tie their payments to certain quality measures. Even then, Jha said, “the broad consensus in the community was that it didn’t really work.” Despite this, the system was implemented on a national scale by the Affordable Care Act (ACA), which created the HVBP program at more than 90 percent of U.S. hospitals. “There was a sense that we needed to begin somewhere,” he explained, “and that this program would be good to test out nationally.”

Medicine and education could learn from one another. Why has there been so little attention to the persistent failure of pay-for-performance plans in education? States and districts and the federal government pour hundreds of millions, billions, into developing incentives, despite the fact that the National Academy of Sciences studied the issue in education and said that such plans don’t work, especially when the bonuses are tied to test scores.

I wrote a response to an editorial that appeared in the Boston Globe, which advocated for using test scores to judge teacher quality.

My response explained why that idea doesn’t work.

I cited evidence and experience.

But people who live in Massachusetts who don’t read the Globe online won’t see it.

Please forward to friends, elected officials, and policymakers.

Open the article to see the links to sources.

Here are some excerpts:

Evaluating teachers by test scores has not raised scores significantly anywhere. Good teachers have been fired by this flawed method. A New York judge ruled this method “arbitrary and capricious” after one of the state’s best teachers was judged ineffective.

Test-based evaluation has demoralized teachers because they know it is unfair to judge them by student scores. Many believe it has contributed to a growing national teacher shortage and declining enrollments in teacher education programs.

A major problem with test-based evaluation is that students are not randomly assigned. Teachers in affluent suburbs may get higher scores year after year, while teachers in urban districts enrolling many high-need students will not see big test score gains. Teachers of English-language learners, teachers of students with cognitive disabilities, and teachers of children who live in poverty are unlikely to see big test score gains, even though they are as good or even better than their peers in the suburbs. Even teachers of the gifted are unlikely to see big test score gains, because their students already have such high scores. Test scores are a measure of class composition, not teacher quality.

Seventy percent of teachers do not teach subjects that have annual tests. Schools could develop standardized tests for every subject, including the arts and physical education. But most have chosen to rate these teachers by the scores of students they don’t know and subjects they never taught.

Scholarly groups like the American Educational Research Association and the American Statistical Association have warned against using test scores to rate individual teachers. There are too many uncontrolled variables, as well as individual differences among students to make these ratings valid. The biggest source of variation in test scores is not the teacher, but students’ family income and home environment.

The American Statistical Association said that teachers affect 1 percent to 14 percent of test score variation. The ASA is an impeccable nonpartisan, authoritative source, not influenced by the teachers’ unions.

The Gates Foundation gave a grant of $100 million to the schools of Hillsborough County, Florida (Tampa), to evaluate their teachers by gains and losses in student test scores. It was an abject failure. The district drained its reserve funds, spending nearly $200 million to implement the foundation’s ideas. Gates refused to pay the last $20 million on its $100 million pledge. The superintendent who led the effort was fired and replaced by one who promised a different direction.

Should Massachusetts cling to a costly, failed, and demoralizing way to evaluate teachers? Should it ignore evidence and experience?

Common sense and logic say no.

Should teachers be judged “subjectively”? Of course. That is called human judgment. Is it perfect? No. Can it be corrected? Yes. Most professionals are judged subjectively by their supervisors and bosses. Standardized tests are flawed instruments. They are normed on a bell curve, guaranteeing winners and losers. They often contain errors — statistical errors, human errors, random errors, scoring errors, poorly worded questions, two right answers, no right answers. No one’s professional career should hinge on the answers to standardized test questions.

Massachusetts is widely considered the best state school system in the nation. The hunt for bad teachers who were somehow undetected by their supervisors is fruitless. The Legislature is right to return the decision about which teachers are effective and which are not to the professionals who see their work every day.

Diane Ravitch is president of the Network for Public Education, a nonprofit that advocates on behalf of public education. She is the author of “Reign of Error: The Hoax of the Privatization Movement and the Danger to America’s Schools.”

Surprise: I wrote the article.

Nothing I wrote will be a surprise to readers of this blog, but may be new to the readers of The Boston Globe.

I was moved to write it because the Globe published an editorial calling for the opposite.

Last week, the Houston Independent School Board deadlocked in a 3-3 tie vote on whether to renew its contract with the vendor supplying the teacher evaluation program.

Audrey Amrein-Beardsley explains their decision here.

At least three board members realized that five years of this program had not moved the needle by an inch. If performance matters, then EVAAS was a failure.

Beardsley is one of the nation’s leading researchers in the study of teacher evaluation.

She writes:

Seven teachers in the Houston Independent School District (HISD), with the support of the Houston Federation of Teachers (HFT), are taking HISD to federal court over how their value-added scores, derived via the Education Value-Added Assessment System (EVAAS), are being used, and allegedly abused, while this district that has tied more high-stakes consequences to value-added output than any other district/state in the nation. The case, Houston Federation of Teachers, et al. v. Houston ISD, is ongoing.

But just announced is that the HISD school board, in a 3:3 split vote late last Thursday night, elected to no longer pay an annual $680K to SAS Institute Inc. to calculate the district’s EVAAS value-added estimates. As per an HFT press release (below), HISD “will not be renewing the district’s seriously flawed teacher evaluation system, [which is] good news for students, teachers and the community, [although] the school board and incoming superintendent must work with educators and others to choose a more effective system.”

Open the link, read the full article, and read her links. This is excellent news.

The bad part of her post is the news that the federal government is still giving out grants that require districts to continue using this flawed methodology, despite the fact that it hasn’t worked anywhere.

Apparently, HISD was holding onto the EVAAS, despite the research surrounding the EVAAS in general and in Houston, in that they have received (and are still set to receive) over $4 million in federal grant funds that has required them to have value-added estimates as a component of their evaluation and accountability system(s).

So Houston will have to find a new vendor of a failed methodology.

The Boston Globe seems to be the Rip van Winkle of the mainstream media. It recently published an editorial that insists that teachers should be evaluated by the test scores of their students. Really. Apparently it is still 2010 in the offices of the Globe, when Arne Duncan claimed that this was the very best way to determine which teachers were effective or ineffective.

 

But it is no longer 2010. The U.S. Departnent of Education handed out $5 billion to states to promote test-based evaluation. The Gates Foundation gave away hundreds of millions of dollars to states to use test scores to evaluate teachers. This method has had negative results everywhere. It has demoralized teachers everywhere. It has contributed to a growing national teacher shortage and declining enrollments in education programs.

 

Scholarly groups like the American Educational Research Association and the American Statistical Association have warned against using test scores to rate individual teachers. There are too many uncontrolled variables, as well as individual differences among students. The American Statistical Association said that teachers affect 1-14% of test score variation. Surely the Boston Globe editorial board must be aware of that report by an impeccable nonpartisan authoritative source. Surely the Boston Globe editorial board must know that teachers in affluent districts are likely to produce high test scores, while teachers of children with disabilities, English language learners, impoverished children, and homeless children are likely to get low test scores. Even teachers of the gifted will receive low ratings because their students get small test score gains since they are already at the top of the scale.

 

The Boston Globe editorial board should learn about the disastrous experience with Gates-style test-based evaluation in Hillsborough County, Florida. The district accepted a $100 million award from the Gates Foundation to rate its teachers by test score gains and losses. It was an abject failure. The district drained its reserve funds. It concluded that it would cost the district $52 million a year to sustain the Gates program. The superintendent who led the effort, MaryEllen Elia, was fired. Gates cut its ties to the county and stopped the payout after wasting $80 million.

 

Should Massachusetts cling to a costly, expensive, failed way to evaluate teachers? Should it ignore evidence and experience?

 

Common sense and logic say no. Will someone send this post to the editorial board of the Boston Globe?

 

 

 

 

 

 

 

 

 

2010 was the high watermark of the corporate reform movement.

In spring 2010, the entire staff at Central Falls, Rhode Island, was fired because of low test scores, which created a national sensation. Arne Duncan and President Obama hailed the courage of Deborah Gist, the state superintendent, and Frances Gallo, the city superintendent, who ordered and confirmed the strategy. Duncan said the firings showed that the administrators were “doing the right things for kids.”

Thus began the reformers’ war against teachers.

In September 2010, “Waiting for Superman,” debuted with a multimillion dollar campaign to promote it: the cover of TIME, appearances by the “stars” on Oprah (Michelle Rhee, Geoffrey Canada, Bill Gates, etc.), and NBC’s Education Nation, focused on promoting the film and its advocacy for charters. “Superman” was a hit job on unions, teachers, and public schools. Its data were skewed, and some of its scenes were staged. It was denied an Academy Award. But Bill Gates put up at least $2 million for public relations.

Thus launched the reformers’ fraudulent fight for privatization as a “civil rights” issue.

Into this fray came the Los Angeles Times, with its own evaluation of thousands of teachers in Los Angeles, created by an economist who employed the methods approved by the Gates Foundation. Teachers were rated on a scale from least effective to most effective. One of those teachers, a dedicated fifth grade teacher named Rigoberto Ruelas, jumped off a bridge and committed suicide after he was publicly labeled as one of the least effective teachers in math and average in reading. Who knew that becoming a teacher would be a hazardous profession?

Anthony Cody delves into the journalistic responsibility of the Los Angeles Times in this important post. The LA Times hired an economist who created VAM ratings and used test scores to rank teachers. Its reporters, Jason Felch and Jason Song, warned against using test scores as the only measure to rank teachers, then proceeded to use test scores as the only measure to rank teachers. The two Jasons, as they were known, hoped to win a Pulitzer Prize. They didn’t. They did come in second in the Education Writers Association choice of the best reporting of the year. Felch was subsequently fired for an ethical breach that involved inappropriate relations with a source.

Cody is concerned about the ethics of journalists who cloak their advocacy and partisanship behind the charade of journalistic independence.

Now, it turns out that the Hechinger Institute at Teachers College, Columbia University, funded the LA Times’ rating scheme. And who do you think funded the Hechinger Report: the Gates Foundation.

We know more about VAM now. We know that it has been rejected by numerous scholars and scholarly associations as invalid, unstable, and unreliable.

Who killed Rigoberto Ruelas?

Anthony Cody here reviews the annual report of the CEO of the Gates Foundation, Sue Desmond-Hellman, and finds it wanting, specifically its lack of humility and its absence of reflection.

 

Of course, Gates will “double down” on Common Core, no matter how many educators call for revisions.

 

But that’s not all. How about some reflection by Gates on the failure of test-based teacher accountability, whether based on “value added” or “student growth”?

 

How about explaining the debacle in Hillsborough County, Florida, which gave up on the Gates initiative after wasting more than $100 million?

 

Why no mention of the foundation’s push for charter schools, which replace public schools and divide communities?

 

Why no candid reflection on the disappointing results of the marketing of more and more technology for the classroom?

 

All in all, a report that shows a megafoundation incapable or unwilling to review its programs with honesty and integrity.

 

 

The Oklahoma legislature passed a law eliminating student test scores as part of teacher evaluation. Hawaii did the same last week. Bit by bit, the most ill-advised, costly, and demoralizing part of Race to the Top is being rejected by the states. It has no research base. Researchers find that measuring teachers by their student scores is unreliable, unstable, and varies by the composition of the class. Its biggest contribution to American education has been to drive out good teachers and create s teacher shortage.

 

House leaders unanimously passed a bill Wednesday that eliminates the requirement to use student academic growth in Oklahoma’s teacher evaluation system.

 

House Bill 2957, which is estimated to save Oklahoma school districts millions of dollars and the Oklahoma State Department of Education more than $500,000, has been sent to the governor’s desk for signature.

 

“Amid this difficult budget year when public education has faced a variety of challenges, House Bill 2957 is a true bright spot of this year’s legislative session,” State Superintendent of Public Instruction Joy Hofmeister said. “By giving districts the option of removing the quantitative portion of teacher evaluations, we not only increase local control but lift outcomes by supporting our teachers while strengthening their professional development and growth in the classroom.”

 

Also praising the bill for its return to local decision-making was Rep. Michael Rogers,R-Broken Arrow, HB 2957’s House author.

 

“This legislation will return flexibility back to the districts on their evaluations while developing an individualized professional development program that will help all of our teachers and administrators,” he said.

 

HB 2957 removes the controversial and mandated Value-Added Measures – which tie a teacher’s performance rating to student test scores — from OSDE’s Teacher and Leader Effectiveness evaluation system and effectively eliminates the requirement that evaluation scores be used to terminate teachers. These quantitative evaluation tools will become optional for districts upon the governor’s signature.

 

Sen. John Ford, R-Bartlesville, who co-authored the bill, said the legislation has been long overdue.

 

“After gathering input from a variety of stakeholders through a lengthy and thoughtful review process, we feel that HB 2957 promotes increased reflection and professional growth for teachers and leaders,” Ford said. “Now is the time to support the teachers in Oklahoma’s public education system by focusing on an evaluation system that places professional development first.”

 

Farewell and good riddance!

 

 

– See more at: http://m.examiner-enterprise.com/news/local-news/lawmakers-pass-teacher-evaluation-changes#sthash.xJo33ldE.dpuf