Archives for category: VAM (value-added modeling)

Audrey Amrein Beardsley is one of the leading experts in the nation in the field of value-added assessment and also one of the nation’s leading skeptics of the claim that teacher “effectiveness” can be measured by the test scores of their students.

Recently, a study was published by economists that purported to measure the effect of teachers’ on their students’ height. The study was a blatant lampoon of VAM (value-added modeling or measurement).

It turns out that I was one of about 25 people who promptly forwarded it to Amrein-Beardsley.

She reviewed the study here. 

Beardsley reminds us of a paper written by economist Jesse Rothstein nearly a decade ago in which he lacerated VAM by showing that it could be twisted to show the effect of teachers on students’ past achievement, a feat that is clearly absurd.

When a policy idea like VAM becomes the target of satire, you know that it is well and truly dead. Now, if only someone would tell the state legislatures that.

A group of scholars collaborated to write a paper published by the National Bureau of Economic Research that studies how teachers affect student height. It is a wonderful and humorous takedown of the Raj Chetty et al thesis that the effects of a single teacher in the early grades may determine a student’s future lifetime earnings, her likelihood graduating from college, live in higher SES neighborhoods, as well as avoid teen pregnancy.

When the Chetty study was announced in 2011, a front-page article in the New York Times said:

WASHINGTON — Elementary- and middle-school teachers who help raise their students’ standardized-test scores seem to have a wide-ranging, lasting positive effect on those students’ lives beyond academics, including lower teenage-pregnancy rates and greater college matriculation and adult earnings, according to a new study that tracked 2.5 million students over 20 years.

The paper, by Raj Chetty and John N. Friedman of Harvard and Jonah E. Rockoff of Columbia, all economists, examines a larger number of students over a longer period of time with more in-depth data than many earlier studies, allowing for a deeper look at how much the quality of individual teachers matters over the long term.

“That test scores help you get more education, and that more education has an earnings effect — that makes sense to a lot of people,” said Robert H. Meyer, director of the Value-Added Research Center at the University of Wisconsin-Madison, which studies teacher measurement but was not involved in this study. “This study skips the stages, and shows differences in teachers mean differences in earnings.”

The study, which the economics professors have presented to colleagues in more than a dozen seminars over the past year and plan to submit to a journal, is the largest look yet at the controversial “value-added ratings,” which measure the impact individual teachers have on student test scores. It is likely to influence the roiling national debates about the importance of quality teachers and how best to measure that quality.

Many school districts, including those in Washington and Houston, have begun to use value-added metrics to influence decisions on hiring, pay and even firing….

Replacing a poor teacher with an average one would raise a single classroom’s lifetime earnings by about $266,000, the economists estimate. Multiply that by a career’s worth of classrooms.

“If you leave a low value-added teacher in your school for 10 years, rather than replacing him with an average teacher, you are hypothetically talking about $2.5 million in lost income,” said Professor Friedman, one of the coauthors…

The authors argue that school districts should use value-added measures in evaluations, and to remove the lowest performers, despite the disruption and uncertainty involved.

“The message is to fire people sooner rather than later,” Professor Friedman said.

Professor Chetty acknowledged, “Of course there are going to be mistakes — teachers who get fired who do not deserve to get fired.” But he said that using value-added scores would lead to fewer mistakes, not more.

President Obama hailed the  Chetty study in his 2012 State of the Union address.

Value-added teacher evaluation, that is, basing the evaluation of teachers on the rise or fall of their students’ test scores, was a central feature of Arne Duncan’s Race to the Top when it was unveiled in 2010. States had to agree to adopt it if they wanted to be eligible for Race to the Top funding.

When the Los Angeles Times published a value-added ranking of thousands of teachers, teachers said the rankings were filled with error, but Duncan said those who complained were afraid to learn the truth. In Florida, teacher evaluations may be based on the rise or fall of the scores of students that the teachers had never taught, in subjects they had never taught. (About 70% of teachers do not teach subjects that are tested annually to provide fodder for these ratings.) When this nutty process was challenged inn court by Florida teachers, the judge ruled that the practice might be unfair but it was not unconstitutional.

The fundamental claim of VAM (value-added modeling or measurement) has been repeatedly challenged, most notably by economist Moshe Adler. When put into law, as it was in most states, it was found to be useless, because only tiny percentages of teachers were identified as ineffective, and even the validity of the ratings of that 1-3% was dubious. The use of VAM was frozen by a judge in New Mexico, then tossed out earlier this year by a new Democratic governor. It was banned by a judge in Houston.  A large experiment funded by the Gates Foundation intended to demonstrate the value of VAM produced negative results.

Now comes economic research to test the validity of linking teacher evaluation and student height.

 

Marianne Bitler, Sean  Corcoran, Thurston Domina, and Emily Penner wrote:

NBER Working Paper No. 26480
Issued in November 2019
NBER Program(s):Program on Children, Economics of Education Program

Estimates of teacher “value-added” suggest teachers vary substantially in their ability to promote student learning. Prompted by this finding, many states and school districts have adopted value-added measures as indicators of teacher job performance. In this paper, we conduct a new test of the validity of value-added models. Using administrative student data from New York City, we apply commonly estimated value-added models to an outcome teachers cannot plausibly affect: student height. We find the standard deviation of teacher effects on height is nearly as large as that for math and reading achievement, raising obvious questions about validity. Subsequent analysis finds these “effects” are largely spurious variation (noise), rather than bias resulting from sorting on unobserved factors related to achievement. Given the difficulty of differentiating signal from noise in real-world teacher effect estimates, this paper serves as a cautionary tale for their use in practice.

 

Mercedes Schneider reviews the Gates Foundation’s long and costly list of failed interventions into K-12 schools and points out, quoting the words of the Foundation, that it has never admitted any failure and never apologized.

Gates paid for the interventions but the real cost was borne by teachers and public schools.

He tried breaking up big schools into small schools, convinced as he was that big schools are ineffective, but when the small schools didn’t produce higher test scores, he abandoned that idea.

He prodded Arne Duncan to include the untested of evaluating teachers by the test scores of their students, and he launched his own experiments in seven districts and charter chains. That too was a flop.

He poured uncounted millions into boosting the charter industry, despite the fact that charters do not get different results from public schools when they enroll the same students.

He spent millions promoting a charter law in his home state of Washington, which passed on the fourth state referendum only after he overwhelmed the opposition by spending 16 times as much as they did; the charters he fought so hard for have struggled to get enough enrollment to stay open (four of the original dozen have already folded), and a CREDO evaluation concluded that they don’t get different results than public schools in the state.

Gates provided almost all the funding necessary for the Common Core State Standards, which required districts and states to spend billions of dollars on new tests, new textbooks, new software, new teacher training, new everything.

When the backlash grew against the Common Core, Gates simply didn’t understand it, since he compares education to an electric plug with standard current into which all possible appliances can be plugged in and get power.

This year, the Gates Foundation awarded 476 grants, but only seven went to K-12, mostly to promote charter schools, a passion he shares with the rightwing Walton Foundation and Betsy DeVos and her foundations.

Read the Gates Foundation’s statement that Mercedes includes in her post. You will see that the foundation acknowledges no failures, no errors, no miscalculations. It doesn’t even own its almost total responsibility for CC, nor for its disastrous reception by teachers and the public.

The legacy of Bill Gates: Teachers and principals who were fired based on a phony measure of their “effectiveness.” Schools in black and brown communities closed because of their test scores. A demoralization of teachers, and a dramatic decline in the number of people entering the profession. A national teacher shortage. The elevation of standardized testing as both the means and the ends of all education (tests that were never used in the schools he and his own children attended).

Here are a few things that Bill Gates NEVER funded or fought for: class size reduction; higher salaries for teachers; a nurse and social worker and librarian in every school; higher taxes to support public schools.

Mercedes concludes:

It may be too much to expect Bill Gates to completely exit K12 education. After all, we have been his hobby for years.

But the fewer Gates dollars, the smaller the petri dish.

Unfortunately the lingering effects of his failed experiments continue to ruin schools, such as the value-added measurement of teachers by test scores, still written into law in many states; the Common Core persists, often under a different name to disguise it; and of course charter schools continue to drain students and resources from underfunded public schools.

 

 

Candidates backed by teachers for local school boards in the Denver area (including Denver) won all but one race.

Candidates supported by the teachers’ unions swept school board elections in Denver, Aurora, Douglas County, Littleton, Adams County, Cherry Creek, and Jeffco. The only candidate to lose was in Jeffco.

The “reformers” bold experiment in teacher-bashing has come to an end, at least for now.

Now it is time for the Colorado legislature to eliminate former state senator Michael Johnston’s failed educator evaluation law, which bases 50% of evaluations on test scores. The law was declared a failure even by its supporters but remains on the books. It did not identify “bad” teachers and it did not produce “great teachers, great principals, or great schools,” as Johnston promised in 2010 when his law was passed.

Reformer Van Schoales wrote in Education Week two years ago:

Colorado Department of Education data released in February show that the distribution of teacher effectiveness in the state looks much as it did before passage of the bill. Eighty-eight percent of Colorado teachers were rated effective or highly effective, 4 percent were partially effective, 7.8 percent of teachers were not rated, and less than 1 percent were deemed ineffective. In other words, we leveraged everything we could and not only didn’t advance teacher effectiveness, we created a massive bureaucracy and alienated many in the field.

The problem, he said, was implementation. Every failed reform is dogged by poor implementation. That’s what they said about the Soviet Union and the Common Core.

When the Waltons paid for an analysis of their failure to pass a referendum to expand charter schools in Massachusetts in 2016, their advisors told them that teachers are trusted messengers. The public likes teachers and believes them. They are more credible than out of state billionaires. The Waltons, too, concluded that the problem with their message was poor implementation, not a rejection by the public, which values its public schools (I go into greater detail in my new book SLAYING GOLIATH, which will be published January 21.)

 

 

 

Every year since 2014, Democrats who fervently support the privatization of public schools have gathered at a conference they pretentiously call “Camp Philos.”

https://campphilos.org/

Check the agenda of meetings present and past.

There you will see the lineup of Democrats who sneer at public schools and look on public school teachers with contempt.

These are the Democrats who support the DeVos agenda of disrupting and privatizing public schools.

They are meeting again this year, and they will slap each other on the back for supporting school closures, charter schools, high-stakes testing, evaluating teachers by the test scores of their students, and hiring inexperienced teachers.

They have the chutzpah to call themselves “stakeholders,” although none of them are teachers, parents of public school students, or have any stake in the public schools that enroll 85-90% of all American students. Exactly what do they have a “stake” in?

 

 

Mr. and Mrs. Bill Gates apparently feel they are not winning enough battles in the court of public opinion, so they have created a lobbying organization to promote their ideas in Congress and state legislatures. 

Will the Gates lobby push for Common Core? For more high-stakes testing? For more federal funding for charter schools? For evaluating teachers by the test scores of their students? For more technology in the classroom?

These are but a few of Bill Gates’ failed education initiatives. Has he learned from failure or will he use his C4 lobby to push his failed ideas even more?

Bill and Melinda Gates have launched a lobbying organization to advocate for issues in health, education, and poverty, The Hill reported on Thursday.

The Gates Policy Initiative, which was announced on Thursday, will work with lawmakers on issues such as global health, global development, moving people from poverty to employment, and education for black, Latino, and rural students. The initiative, which will be a 501(c)(4) organization under the US tax code, is independent from the Bill and Melinda Gates Foundation, the billionaire couple’s philanthropic organization.

Rob Nabors, the director of the Bill and Melinda Gates Foundation and the former White House director of legislative affairs during the Obama administration, told The Hill that the Gates Policy Initiative would work in a bipartisan way.

In an article in The Hill, Rob Nabors said the new lobbying organization would reflect the work of the foundation.

Much of what they’ve learned running their foundation will help them through the process of establishing a lobbying shop.

“Probably the most important point for us is similar to the way Bill and Melinda have approached their philanthropic giving and other things that they do. They are interested in learning what works and what doesn’t work,” Nabors said.

He said that if they are not successful in a couple of years, they will “shutter the shop and figure out what else could potentially be done.”

“I think that experimental type of approach, that innovative type of approach, is both relatively unique in this space and embedded into the DNA that Bill and Melinda bring with them,” he said.

Nabors said that when he worked in the Obama White House, his job was often described as the White House chief lobbyist.

“I’m excited to get back into the mix of talking to people specifically about the work that they are doing every day, trying to put bills together that will make people’s lives better,” he said.

He added that Bill and Melinda Gates also bring a unique lens to a lobbying shop.

“They are very data-focused so a number of the types of issues that we will be exploring and the solutions that we are exploring are based on data that we collected from programs that we funded,” he added.

 

The battle has begun about whether to lift the cap on charter schools in New York City.

New York City has 235 charter schools serving 123,000 students (about 10% of those enrolled in public schools) and there are no empty slots for additional charters unless the legislature raises the cap.  Governor Cuomo, flush with hedge fund cash from his last campaign, wants to raise the charter cap.

Now billionaire Merryl Tisch, who previously was Chancellor of the New York Board of Regents and is now on the board of the State University  of New York, proposed that the city be allowed to use some of the 99 open charter slots from the rest of the state. 

Under Tisch’s leadership at the Regents, New York won a Race to the Top grant of $700 million, hired John King as State Commissioner, committed to evaluate teachers by the test scores of their students, and adopted the Common Core and PARCC Testing. Tisch set off the Opt Out Movement, and she also hired MaryEllen Elia from Hillsborough County in Florida, which was part of the Gates Foundation’s failed experiment with VAM (value-added measurement) of teachers.

We are told that the waiting list for admission to charters in NYC is very, very long.

So think about this:

If there is a long waiting list, as Merryl Tisch says, why do charters hire a marketing firm to send recruiting letters to children in public schools? Why are they moaning about not having access to the public school names and addresses? Why don’t they just accept kids from the waiting list? Is there a waiting list? Maybe there are actually vacancies, as in Los Angeles, where 80% of the charters have empty seats. Even Eva Moskowitz needs access to public school names for recruitment purposes.

Would someone please audit that alleged waiting list?

 

 

 

 

William Sanders was an agricultural statistician who developed a secret, patented formula for measuring teacher effectiveness. It’s call EVASS. It was tossed out by a Houston judge who said it was wrong to judge teachers by a secret algorithm that they could neither examine nor question.

As Stuart Egan reports, North Carolina clings to EVASS, no matter how many times it has been discredited (by scholars such as Audrey Amerein-Beardsley) or by courts that findit arbitrary and inscrutable.

Want to understand how teachers in North Carolina are evaluated?

Egan writes:

“Actually, the chain is from a .gov to a .org to a .com.”

Rupert Murdoch’s New York Post has an editorial chastising the New York Board of Regents for failing to reactivate the value-added method of evaluating teachers by the test scores of their students.

Whoever wrote the editorial is ignorant of the fact that the American Statistical Association warned against the use of this method or the RAND-AIR report that found zero benefit from the Gates Foundation’s investment in districts that used this method.

This teacher-blogger reminds readers that a New York Judge already labeled this method “junk Science.”

The Network for Public Education Action Fund is delighted to endorse new leadership for New Mexico: Michelle Lujan Grisham for Governor and Howie Morales for Lieutenant Governor. After eight years of horrible education policies, Lujan and Morales wupill be a breath of fresh air for students and teachers. The Land of Enchantment has one of the highest rates of child poverty in the nation, which the previous administration ignored. Instead, it insisted on high-stakes teacher evaluations, which are currently enjoined by court order. Despite—or because—of eight years of failed Reform policies, New Mexico remains stuck at the bottom of NAEP.

Time for new thinking!

The Network for Public Education Action has endorsed Congresswoman Michelle Lujan Grisham for Governor of New Mexico and State Senator Howie Morales for Lieutenant Governor.

Grisham, a 12th-generation New Mexican, has served as the U.S. Representative for New Mexico’s 1st congressional district since 2013.

Morales is the State Senator for District 28 in the New Mexico Senate. He has an M.A. in bilingual special education and a Ph.D. in curriculum and instruction.

Both candidates have worked to make positive changes in our public schools. As the Secretary of New Mexico’s Department of Health, Grisham expanded the number of school-based health centers in the state. Morales spent 10 years as a special education teacher and was the head baseball coach in the Cobre Consolidated School District.

Grisham and Morales have promised to “end use of the PARCC exam in favor of less intrusive and frequent alternatives, implement authentic and useful assessments developed by teachers to connect with what students are really learning, and reform school and teacher evaluations to focus on more holistic measures of progress.”

When it comes to other critical issues facing New Mexico’s public schools, they have said they will increase funding and make universal access to high-quality Pre-K a reality for every New Mexico family. To address the state’s severe teacher shortage, they intend to support public school employees by raising salaries across the board, including the salaries of assistants and support staff.

On November 6th, please be sure to cast your ballot for these pro-public education candidates.