Archives for category: Teacher Evaluations

The National Center for Research on Education Access and Choice (REACH), formerly known as the Education Research Alliance, released its first report after having been funded by Betsy DeVos with $10 million to study the effects of choice in schools. REACH used value-added methodology (judging teachers by the test score gains of their students to determine that those who got the highest VAM scores were likeliest to stay. It is safe to assume that these teachers were in the highest-scoring charter schools. On the other hand, the teachers with the lowest scores (no doubt, in the lowest-performing schools) were turning over at a high rate. The study’s conclusion is that (some) charters are keeping their best teachers (those with the highest VAM ratings) but (some) charters are not, which since they don’t get high VAM scores, is not a big deal.

We are excited to announce the release of the first study from the National Center for Research on Education Access and Choice (REACH). Naturally, the subject of this study is one that’s considered the most important factor in school success: teachers.

New Orleans is the first all-charter school district in the country. This makes the city the first where schools are held strictly accountable for performance, where many employers in close proximity compete for teachers, and where schools have the ability to respond to these pressures with almost complete autonomy over school personnel. If school reform advocates are right, we would expect these policy changes to produce major change in the teacher labor market. Did this happen?

To answer this question, researchers Nathan Barrett, Deven Carlson, Douglas N. Harris, and Jane Arnold Lincove compared New Orleans to similar neighboring districts from 2010 to 2015, using student test score growth to measure teacher performance. They drew the following conclusions:

Teacher retention is more closely related to teacher performance in New Orleans than in traditional public school districts. Lower performing teachers in New Orleans are 2.5 times more likely to leave their school than high-performing teachers, compared with only 1.9 times in similar neighboring districts.
The stronger link between retention and performance might imply that teacher quality would improve faster in New Orleans than in similar districts. However, this is not the case. The difference in average teacher performance between New Orleans and comparison districts remained essentially unchanged between 2010 and 2015. This is apparently because of the larger share of new teachers in New Orleans, whose lower quality roughly offsets the city’s advantages in retaining higher performing teachers.
The stronger retention-performance link in New Orleans is somewhat related to financial rewards, though not in a way that is likely to increase the overall quality of teaching. We find that higher performing teachers only receive pay increases when they switch schools, which may increase teacher turnover. High-performing teachers do not receive raises for performance when they stay in the same school.
These findings highlight the complexities of policies intended to increase the quality of teaching. Future studies will build on this work by examining how performance-based school closures affect the teacher labor market.

Read the policy brief here or the full technical report here.

Thomas Good sent me this research paper about teacher evaluation that he wrote with Alyson Lavigne.

Division 15 (Educational Psychology) of the American Psychological Association is proud to announce their second policy brief, “Addressing Teacher Evaluation Appropriately.” This brief, focused on teacher evaluation practices and policies in schools was written by Alyson Lavigne and Thomas Good. A copy of the brief is attached for you to read and share.

About the Brief: In this policy brief, Lavigne and Good argue that the most commonly used practices to evaluate teachers—statistical approaches to determine student growth like value-added measures and the observation of teachers—have not improved teaching and learning in U.S. schools. They have not done so because these approaches are problematic, including the failure to adequately account for context, complexity, and that teacher effectiveness and practice varies. With these limitations in mind, the authors provide recommendations for policy and practice, including the elimination of high-stakes teacher evaluation and a greater emphasis on formative feedback, allowing more voice to teachers and underscoring that improving instruction should be at least as important as evaluating instruction.

Share the Brief! It’s important that our national policy be based on sound evidence. We have attached a copy of the brief so that you may share this directly with your constituents—local policymakers, practitioners, educational organizations, faculty, staff, and students who are engaged in K-12 settings and research. You can also promote this important work via social media using Twitter or Facebook using the following link: EdPsych.us/AddressingTeacherEvaluation

If you have any questions about the contents of this brief, please contact Alyson Lavigne (alyson.lavigne@usu.edu). Any questions or ideas for future Division 15 policy briefs should be directed to Sharon Nichols, Chair of Division 15’s Policy and Practice Committee (Sharon.Nichols@utsa.edu). For additional information about research related to problems involved in current teacher evaluation practices, see Lavigne and Good’s recent publication, Enhancing Teacher Education, Development, and Evaluation.

You can read the report here.

Say this for Eric Hanushek: He never gives up on his obsession with paying teachers more if their students get higher test scores. Arne Duncan built this concept into the requirements of his disastrous Race to the Top” program, which caused almost every state to adopt a teacher evaluation plan in which student test scores played a significant role. Harvard economist Raj Chetty wtote a highly-publicized paper with two colleagues, claiming that one good teacher (who raised test scores in the early grades) would raise lifetime incomes (by about $5 a week), reduce pregnancies, and be a life-changer. President Obama cited Chetty in his 2012 State of the Union address, but efforts to turn the theory into reality fell flat. (Read more about this catastrophe in SLAYING GOLIATH.) In fact, every state that imposed value-added measurement learned that it discouraged teachers from teaching in high-needs schools, where their chance of getting a big test score gain was reduced. It did not produce any of the promised benefits.

But forget about reality! Let’s stand by the theory. Hanushek’s new venture at the conservative Hoover Institution is joined by Christopher Ruszkowski, who served as Commissioner of Education in New Mexico after the resignation of Hanna Skandera (who previously worked for the Hoover Institution, Jeb Bush and Arnold Schwarzenegger). After eights years of “reform” leadership, New Mexico remained mired at the bottom of NAEP. The state had a harsh, test-based teacher evaluation plan, but the union fought it in court, it was enjoined by a judge, and the New Democratic Governor scrapped it as one of her first executive actions. New Mexico has one of the highest proportions of students living in poverty, but Republican state leaders ignored that inconvenient fact. After a decade of consistent failure, we can safely put test-based teacher evaluation into the category of a Zombie idea. Dead but still stalking the land.

 

Hoover_Centennial_Logo_RGB Match PMS 202 (red)_w 600

FOR IMMEDIATE RELEASE

CONTACT:

Hoover Institution, Jeff Marschner, (202) 760-3200

NEWLY FORMED HOOVER EDUCATION SUCCESS INITIATIVE RELEASES PAPER ON TRANSFORMING TEACHER COMPENSATION

Four education policy papers to be released in 2020—addressing how states should consider transforming education in the decade ahead.

STANFORD, CA. (January 30th) – As state legislative sessions begin around the country, the Hoover Education Success Initiative (HESI), a new research program at Stanford University’s Hoover Institution, has released “The Unavoidable: Tomorrow’s Teacher Compensation”—a policy briefing on the important connections between teacher compensation systems and student achievement outcomes. The research-based policy paper includes both a summary of findings and practical recommendations for policymakers.

The paper highlights often overlooked areas for attention including shifting overall compensation from retirement into salaries, ending the practice of paying for advanced degrees that do not yield changes in student outcomes, addressing teacher shortages in a targeted fashion instead of generally, and paying teachers more when they are effective in higher-need schools.  The paper concludes that teachers’ salaries should be significantly increased, but that students will not make achievement gains unless salaries are also linked to teacher quality.

“We need to pay teachers competitively, which we are not doing now,” said Dr. Eric Hanushek, author of the policy synthesis. “But just increasing compensation without recognizing teacher effectiveness is unlikely to lead to improved student outcomes. We should bundle together better pay with a serious recognition of just how important effective teachers are when it comes to influencing student achievement.”

“While we have spent much of the last year reviewing and synthesizing the research, the next phase of our work turns to helping states implement the policy ideas,” said Christopher N. Ruszkowski, executive director of HESI. “There is overwhelming evidence that nothing matters more than teacher quality, and state legislatures and governors should take strong action. Neglecting this responsibility causes harm to our students that may not be immediately visible today but will certainly be reflected in our students’ lives and in our economy tomorrow.  It’s a tough issue and it may feel like something we can avoid, but it will catch up with us.”

Click here to read the policy analysis brief.

About the Hoover Education Success Initiative

With passage in 2015 of the Every Student Succeeds Act (ESSA), states are again in charge of American education policy. To support them in this undertaking, the Hoover Education Success Initiative (HESI), launched in 2019, seeks to provide state education leaders with policy recommendations that are based upon sound research and analysis.  HESI hosts workshops and policy symposia on high-impact areas related to the improvement and reinvention of the US education system. The findings and recommendations in each area are outlined in concise topical papers.

The leadership team at HESI engages with its Practitioner Council, formed of national policy leaders, and with interested state government leaders. HESI’s ultimate goal is to spark innovation and contribute to the ongoing transformation of the nation’s K-12 education landscape, thus improving outcomes for our nation’s children.

###

Jeff Marschner
Director of Media Relations

https://www.educationdive.com/news/is-edtpa-standing-in-the-way-of-getting-more-teachers-into-classrooms/572969/

Educators disagree about the value, validity, and reliability of the Pearson EdTPA, which is mandated in many states as the gateway to entering teaching.

Some states have lowered the passing score. Some are wondering whether to abandon it.

The debate occurs at a time when enrollments in teacher education programs have dropped by a third.

While many agree on the importance of high standards for new teachers, it’s by no means clear that the EdTPA encourages better teaching or merely rewards teachers who are good at the demands made by Pearson.

Hi, Bill and Melinda,

We have never met but I feel that I know you because I am so familiar with your education projects.

I have tried in the past to meet you and have a candid conversation but have never had any luck.

You were always too busy or out of town.

But I am trying again.

I will be in Seattle on February 3-4.

I arrive on the afternoon of the 3rd and am speaking at a public event on February 4 at Town Hall. The wonderful teacher-leader Jesse Hagopian is introducing me.

I have some down time and wondered if we might be able to meet at last.

Are you available to meet in the late afternoon or evening of February 3 or during the day on February 4?

Please let me know if you can make time on your busy schedule.

My partner will be with me.

I hope you can do it!

We have a lot to talk about!

Diane

Audrey Amrein Beardsley is one of the leading experts in the nation in the field of value-added assessment and also one of the nation’s leading skeptics of the claim that teacher “effectiveness” can be measured by the test scores of their students.

Recently, a study was published by economists that purported to measure the effect of teachers’ on their students’ height. The study was a blatant lampoon of VAM (value-added modeling or measurement).

It turns out that I was one of about 25 people who promptly forwarded it to Amrein-Beardsley.

She reviewed the study here. 

Beardsley reminds us of a paper written by economist Jesse Rothstein nearly a decade ago in which he lacerated VAM by showing that it could be twisted to show the effect of teachers on students’ past achievement, a feat that is clearly absurd.

When a policy idea like VAM becomes the target of satire, you know that it is well and truly dead. Now, if only someone would tell the state legislatures that.

Alan Singer calls out Common Core for the poor showing of US students on PISA. 

Remember all the promises about how Common Core would raise all test scores and close gaps? Nada.

Of course, the deeper issue is that decades of test-and-punish reforms failed, not just Common Core.

it those who pushed these failed policies will not abandon them. They will say—they are saying—that we must double down on failure.

The consensus among governors and policy elites that followed “A Nation at Risk” in 1983 was that common standards, tests, and accountability would lead to high levels of performance (ie, test scores).

They didn’t. They haven’t. They won’t.

Almost four decades later, we can safely say that this theory of reform has failed. Billions of dollars wasted!

NBCT high school teacher and blogger Justin Parmenter discovered a shocking fact: a company in the state called SAS pays to send state legislators to the annual conference of the American Legislative Exchange Council (ALEC), a far-right anti-public school organization that writes model legislation. SAS sells software to districts and states to evaluate teacher effectiveness.  The SAS software is very controversial because it’s algorithms are secret and proprietary. Teachers in Houston sued and won a court judgement against SAS, when the judge ruled that its secret processes were arbitrary and denied due process to teachers, who had no way to know how they were judged or if the calculations were accurate.

Parmenter writes:

The American Legislative Exchange Council (ALEC) is an infamous legislation factory which is notoriously hostile toward traditional public schools.  Its model bills are passed into law–often word for word–by state legislatures around the country.

ALEC’s education platform claims the nation’s K-12 education system is “failing our students, leaving them unprepared for college, careers, or life,” and the policies the organization writes for lawmakers offer a smorgasbord of legislative pathways for defunding public schools, especially those that serve high-poverty students.

That’s why it’s so disappointing to learn that one of North Carolina K-12 public education’s most high-profile partners, SAS Institute, is paying for members of North Carolina’s General Assembly to travel to ALEC’s annual meetings, where viewing and discussing the group’s suggested anti-public school policies is one of the primary activities.

SAS Institute is a privately held analytics software company based in Cary, NC.  Its founder and CEO James Goodnight’s net worth is estimated at more than $13 billion, making him the richest person in North Carolina by a wide margin.

SAS has an extremely cozy relationship with the NC Department of Public Instruction (DPI).  Just last month, for example, SAS hosted an event where company software specialists and professional educators including DPI Deputy Superintendent of District Support Dr. Beverly Emory presented on how to use SAS data in public schools.

Millions of North Carolina taxpayer dollars go to SAS every year for a variety of software-related education contracts.  The company provides K-12 teachers with EVAAS ratings, which employ a secret algorithm to measure individual teachers’ effectiveness using DPI’s standardized test data.  It also produces the North Carolina School Report Cards.

North Carolina’s School Report Cards assign each school a single A-F letter grade representing its overall performance. The report cards have been controversial since state legislators introduced them in 2013 as the grades are highly correlated with levels of poverty and sometimes have the effect of pushing families away from traditional public schools.

Probably not by coincidence, ALEC has been peddling its “A-Plus Literacy Act” to lawmakers since early 2011.  The model bill recommends a statewide A-F school report card system with a special focus on reporting results for students who score in the lowest 25th percentile, and it refers to the grading system as a “lynchpin for reforms.”  One such reform is also included in the bill, as ALEC recommends students who attend F schools be given an opportunity to enroll in private schools instead.

A cozy arrangement indeed!

A group of scholars collaborated to write a paper published by the National Bureau of Economic Research that studies how teachers affect student height. It is a wonderful and humorous takedown of the Raj Chetty et al thesis that the effects of a single teacher in the early grades may determine a student’s future lifetime earnings, her likelihood graduating from college, live in higher SES neighborhoods, as well as avoid teen pregnancy.

When the Chetty study was announced in 2011, a front-page article in the New York Times said:

WASHINGTON — Elementary- and middle-school teachers who help raise their students’ standardized-test scores seem to have a wide-ranging, lasting positive effect on those students’ lives beyond academics, including lower teenage-pregnancy rates and greater college matriculation and adult earnings, according to a new study that tracked 2.5 million students over 20 years.

The paper, by Raj Chetty and John N. Friedman of Harvard and Jonah E. Rockoff of Columbia, all economists, examines a larger number of students over a longer period of time with more in-depth data than many earlier studies, allowing for a deeper look at how much the quality of individual teachers matters over the long term.

“That test scores help you get more education, and that more education has an earnings effect — that makes sense to a lot of people,” said Robert H. Meyer, director of the Value-Added Research Center at the University of Wisconsin-Madison, which studies teacher measurement but was not involved in this study. “This study skips the stages, and shows differences in teachers mean differences in earnings.”

The study, which the economics professors have presented to colleagues in more than a dozen seminars over the past year and plan to submit to a journal, is the largest look yet at the controversial “value-added ratings,” which measure the impact individual teachers have on student test scores. It is likely to influence the roiling national debates about the importance of quality teachers and how best to measure that quality.

Many school districts, including those in Washington and Houston, have begun to use value-added metrics to influence decisions on hiring, pay and even firing….

Replacing a poor teacher with an average one would raise a single classroom’s lifetime earnings by about $266,000, the economists estimate. Multiply that by a career’s worth of classrooms.

“If you leave a low value-added teacher in your school for 10 years, rather than replacing him with an average teacher, you are hypothetically talking about $2.5 million in lost income,” said Professor Friedman, one of the coauthors…

The authors argue that school districts should use value-added measures in evaluations, and to remove the lowest performers, despite the disruption and uncertainty involved.

“The message is to fire people sooner rather than later,” Professor Friedman said.

Professor Chetty acknowledged, “Of course there are going to be mistakes — teachers who get fired who do not deserve to get fired.” But he said that using value-added scores would lead to fewer mistakes, not more.

President Obama hailed the  Chetty study in his 2012 State of the Union address.

Value-added teacher evaluation, that is, basing the evaluation of teachers on the rise or fall of their students’ test scores, was a central feature of Arne Duncan’s Race to the Top when it was unveiled in 2010. States had to agree to adopt it if they wanted to be eligible for Race to the Top funding.

When the Los Angeles Times published a value-added ranking of thousands of teachers, teachers said the rankings were filled with error, but Duncan said those who complained were afraid to learn the truth. In Florida, teacher evaluations may be based on the rise or fall of the scores of students that the teachers had never taught, in subjects they had never taught. (About 70% of teachers do not teach subjects that are tested annually to provide fodder for these ratings.) When this nutty process was challenged inn court by Florida teachers, the judge ruled that the practice might be unfair but it was not unconstitutional.

The fundamental claim of VAM (value-added modeling or measurement) has been repeatedly challenged, most notably by economist Moshe Adler. When put into law, as it was in most states, it was found to be useless, because only tiny percentages of teachers were identified as ineffective, and even the validity of the ratings of that 1-3% was dubious. The use of VAM was frozen by a judge in New Mexico, then tossed out earlier this year by a new Democratic governor. It was banned by a judge in Houston.  A large experiment funded by the Gates Foundation intended to demonstrate the value of VAM produced negative results.

Now comes economic research to test the validity of linking teacher evaluation and student height.

 

Marianne Bitler, Sean  Corcoran, Thurston Domina, and Emily Penner wrote:

NBER Working Paper No. 26480
Issued in November 2019
NBER Program(s):Program on Children, Economics of Education Program

Estimates of teacher “value-added” suggest teachers vary substantially in their ability to promote student learning. Prompted by this finding, many states and school districts have adopted value-added measures as indicators of teacher job performance. In this paper, we conduct a new test of the validity of value-added models. Using administrative student data from New York City, we apply commonly estimated value-added models to an outcome teachers cannot plausibly affect: student height. We find the standard deviation of teacher effects on height is nearly as large as that for math and reading achievement, raising obvious questions about validity. Subsequent analysis finds these “effects” are largely spurious variation (noise), rather than bias resulting from sorting on unobserved factors related to achievement. Given the difficulty of differentiating signal from noise in real-world teacher effect estimates, this paper serves as a cautionary tale for their use in practice.

 

Mercedes Schneider reviews the Gates Foundation’s long and costly list of failed interventions into K-12 schools and points out, quoting the words of the Foundation, that it has never admitted any failure and never apologized.

Gates paid for the interventions but the real cost was borne by teachers and public schools.

He tried breaking up big schools into small schools, convinced as he was that big schools are ineffective, but when the small schools didn’t produce higher test scores, he abandoned that idea.

He prodded Arne Duncan to include the untested of evaluating teachers by the test scores of their students, and he launched his own experiments in seven districts and charter chains. That too was a flop.

He poured uncounted millions into boosting the charter industry, despite the fact that charters do not get different results from public schools when they enroll the same students.

He spent millions promoting a charter law in his home state of Washington, which passed on the fourth state referendum only after he overwhelmed the opposition by spending 16 times as much as they did; the charters he fought so hard for have struggled to get enough enrollment to stay open (four of the original dozen have already folded), and a CREDO evaluation concluded that they don’t get different results than public schools in the state.

Gates provided almost all the funding necessary for the Common Core State Standards, which required districts and states to spend billions of dollars on new tests, new textbooks, new software, new teacher training, new everything.

When the backlash grew against the Common Core, Gates simply didn’t understand it, since he compares education to an electric plug with standard current into which all possible appliances can be plugged in and get power.

This year, the Gates Foundation awarded 476 grants, but only seven went to K-12, mostly to promote charter schools, a passion he shares with the rightwing Walton Foundation and Betsy DeVos and her foundations.

Read the Gates Foundation’s statement that Mercedes includes in her post. You will see that the foundation acknowledges no failures, no errors, no miscalculations. It doesn’t even own its almost total responsibility for CC, nor for its disastrous reception by teachers and the public.

The legacy of Bill Gates: Teachers and principals who were fired based on a phony measure of their “effectiveness.” Schools in black and brown communities closed because of their test scores. A demoralization of teachers, and a dramatic decline in the number of people entering the profession. A national teacher shortage. The elevation of standardized testing as both the means and the ends of all education (tests that were never used in the schools he and his own children attended).

Here are a few things that Bill Gates NEVER funded or fought for: class size reduction; higher salaries for teachers; a nurse and social worker and librarian in every school; higher taxes to support public schools.

Mercedes concludes:

It may be too much to expect Bill Gates to completely exit K12 education. After all, we have been his hobby for years.

But the fewer Gates dollars, the smaller the petri dish.

Unfortunately the lingering effects of his failed experiments continue to ruin schools, such as the value-added measurement of teachers by test scores, still written into law in many states; the Common Core persists, often under a different name to disguise it; and of course charter schools continue to drain students and resources from underfunded public schools.