Archives for category: Teacher Evaluations
« Older Entries
Newer Entries »

This is a Classic: The American Statistical Association Statement on Value-Added Measurement

By dianeravitch
October 5, 2016 //
21

When this statement first appeared in 2014, I said at the time that it should be on the bulletin board of every public school.

The American Statistical Association explains here why the evaluations of individual teachers should not be based on their students’ test scores.

Here is an excerpt. Read the whole statement, which is only 8 pages long:

It is unknown how full implementation of an accountability system incorporating test-based indicators, such as those derived from VAMs, will affect the actions and dispositions of teachers, principals and other educators. Perceptions of transparency, fairness and credibility will be crucial in determining the degree of success of the system as a whole in achieving its goals of improving the quality of teaching. Given the unpredictability of such complex interacting forces, it is difficult to anticipate how the education system as a whole will be affected and how the educator labor market will respond. We know from experience with other quality improvement undertakings that changes in evaluation strategy have unintended consequences. A decision to use VAMs for teacher evaluations might change the way the tests are viewed and lead to changes in the school environment. For example, more classroom time might be spent on test preparation and on specific content from the test at the exclusion of content that may lead to better long-term learning gains or motivation for students. Certain schools may be hard to staff if there is a perception that it is harder for teachers to achieve good VAM scores when working in them. Overreliance on VAM scores may foster a competitive environment, discouraging collaboration and efforts to improve the educational system as a whole.

Research on VAMs has been fairly consistent that aspects of educational effectiveness that are measurable and within teacher control represent a small part of the total variation in student test scores or growth; most estimates in the literature attribute between 1% and 14% of the total variability to teachers. This is not saying that teachers have little effect on students, but that variation among teachers accounts for a small part of the variation in scores. The majority of the variation in test scores is attributable to factors outside of the teacher’s control such as student and family background, poverty, curriculum, and unmeasured influences.

The VAM scores themselves have large standard errors, even when calculated using several years of data. These large standard errors make rankings unstable, even under the best scenarios for modeling. Combining VAMs across multiple years decreases the standard error of VAM scores. Multiple years of data, however, do not help problems caused when a model systematically undervalues teachers who work in specific contexts or with specific types of students, since that systematic undervaluation would be present in every year of data.

Despite the warning from ASA, which has no special interest and does not represent teachers or public school administrators, many states continue to use this method (called VAM, or value-added measurement or value-added modeling).

States were coerced into adopting this unproven method by the U.S. Department of Education, which said that states had to adopt it if they wanted to be eligible to compete for nearly $5 billion in federal funds in 2009, as every state was undergoing a budget crisis caused by the economic meltdown of fall 2008.

Many states adopted it, and it has not had positive effects in any state.

In Colorado and New York, among others, VAM scores count for as much as 50% of teachers’ evaluation.

A state court in New York ruled this method “arbitrary and capricious” when challenged by fourth grade teacher Sheri Lederman and her lawyer-husband Bruce Lederman.

Some states assign VAM scores to teachers based on students they never taught in subjects they don’t teach.

This is an example of federal and state policy that has no basis in evidence and that has harmed the lives of many teachers. It very likely has caused teachers to leave the profession and contributed to teacher shortages.

Categories Race to the Top, Teacher Evaluations, VAM (value-added modeling)

Should Teachers Be Evaluated by Test Scores? Does One Teacher Affect Your Lifetime Earnings?

By dianeravitch
September 23, 2016 //
42

This post, with an anonymous author, reviews the research on value-added measurement, with frequent references to those who claim that the rise or fall of test scores is the best way to judge teacher quality.

 

The basic question he or she addresses is whether the actions of your kindergarten teacher or your third-grade teacher can affect your lifetime earnings, as Raj Chetty and his team asserted in a study a few years back.

 

The author goes into a lengthy back-and-forth about whether such claims make sense.

 

But the one essential fact that his post is missing is that 70% of teachers do not teach tested subjects. A district or school can evaluate teachers with VAM only when there are enough years of test scores to document the effects of the teacher over several years. Teachers of subjects other than reading and mathematics in grades 3-8 will never get VAM ratings.

 

But many states have solved this problem by assigning VAM ratings to the 70%. Their ratings are based on the scores of students they never met in subjects they never taught. This is called an “attributed rating.”

 

That makes sense, said no one ever.

 

That may be why Hawaii and Oklahoma have dropped VAM. It is expensive and gives false positives and false negatives. Expect more states to join these two states.

Categories Teacher Evaluations, VAM (value-added modeling)

Hawaii: Sorry, But the Nutty Teacher Evaluations Are Still Imposed 

By dianeravitch
September 10, 2016 //
22

Earlier today, I posted a piece by Andy Jones, a high school language arts teacher in Hawaii. In the introduction, I mistakenly said that Hawaii had eliminated the teacher evaluation system that was created in response to winning Race to the Top funding. I wrote: “This past spring, Hawaii dropped the test-based teacher evaluation that Race to the Top had forced on the state as a condition of winning RTTT funds. The money was all gone, and so was this bad idea.” That was not completely accurate.

Andy Jones sent this note to correct me:

“I so hate to ask, but…the HSTA secretary, who along with everyone else here is a great admirer of your work, just asked me if I could mention to you that your intro to my article is somewhat inaccurate, in that the teacher evaluation system (EES) is still in place. We were merely successful in eliminating test scores from the evaluation. We are, however, almost equally angry at present, because in the months since that announcement was made, they have insisted on maintaining the excoriated SLO component (Student Learning Outcomes) – the mindbogglingly ridiculous quasi-action-research template we have to go through in our evaluations in which we choose learning objectives, administer pre-assessments, make predictions about student learning, collect evidence of student growth, etc., and are only rated “distinguished” if 90% of our predictions are accurate. This is now worth 50% of our evaluation, along with the standard Danielson-based observation cycle and Core Professionalism (basically a portfolio on Danielson Domain 4) comprising the other 50%.

“HSTA secretary Amy Perruso wanted me to post you on this, because she’s afraid that HSTA will now be bombarded with emails and telephone calls asking “How did you get rid of the teacher evaluations?” – which, unfortunately we have not and which we’re going to be attempting very aggressively this year to get rid of.

– Andy”

Categories Education Reform, Hawaii, Race to the Top, Teacher Evaluations

Andy Jones: Hawaii’s Education Future Is Looking Brighter

By dianeravitch
September 10, 2016 //
1

Andy Jones is a high school language arts teacher in Hawaii. The public school teachers in Hawaii have been energetic in stopping the worst extremes of the Race to the Top grant that the state received. This past spring, Hawaii dropped the test-based teacher evaluation that Race to the Top had forced on the state as a condition of winning RTTT funds. The money was all gone, and so was this bad idea.

Andy knows the history of American education, the cycles of criticism that follow one another, and he believes that the new federal law will give the state the ability to chart a better course than the one imposed by the Bush-Obama agenda of test and punish. I regret to say that his article is behind a pay wall, but I will share with you what he shared with me. In this time of doom-and-gloom, it is great to hear an optimistic forecast!

Andy writes:


Alarm, overhaul, stagnation, denial, recognition. Repeat ad infinitum.

Students of educational history will recognize in these five words the cycle that has defined American school culture for decades.

Thirty-three years ago, the “A Nation at Risk” report rang the alarm of educational decline. Its clarion call for improved public education resounded for the next generation and led eventually to initiatives such as No Child Left Behind and Race to the Top – programs that inaugurated sweeping, possibly irreversible changes to schools and school communities across the country.

A consensus has now emerged that these changes have led to dismal failure — a consensus signaled by the Every Student Succeeds Act (ESSA), which emphatically seeks to reverse the damage done, in part by giving states back the freedom to define and enact their own vision of 21st-century education.

On the heels of ESSA and the widespread discussion it has initiated, the National Conference of State Legislatures (NCSL) has released a report that may prove as decisive as “A Nation at Risk.”

The title — “No Time to Lose: How to Build a World-Class Education System State by State” — is misleading in that it seems to announce a mere repeat of the alarmist tone of the “Nation” report, perhaps to be followed by a new round of dubious policy suggestions from non-educators.

However, in what must come as a welcome shock to educators accustomed to routine governmental denial of policy failure, “No Time to Lose” fully acknowledges the mistakes of the past 15 years and seconds the sustained criticisms of prominent researchers such as Diane Ravitch and Pasi Sahlberg. These and many others have analyzed the extensive Organisation for Economic Co-operation and Development reports on international education and have concluded that the misguided “reforms” of the past years have had an overwhelmingly negative impact on American schools, leading to ever further decline internationally.

They have also highlighted an additional, sinister aspect of these “reforms,” which have involved the gradual removal of educational decisions from the purview of teachers and educators and the corresponding enrichment of educa- tional corporations profiting from the proliferation of mediocre materials and programs that schools are forced to use.

We are fortunate to be living in a state led by a governor who recognizes what is at stake and who has created a robust task force that is working to establish grassroots consensus as to what is best for Hawaii schools and the students they serve.

We are also fortunate to have an increasingly dynamic teachers union that has sponsored a teacher- written report, “Schools Our Keiki Deserve,” which echoes the advice of our top educational researchers as well as the urgent tone of “No Time to Lose.”

Hawaii Department of Education (DOE) officials have shown signs recently that they are beginning to veer away from the pattern of denial that for years has characterized state and district education departments across the country. They have, for instance, conceded the unhealthy aspects of standardized testing, and they have also begun to embrace the idea of whole-child education as practiced in the world’s top-performing school systems.

As the DOE continues revising the Strategic Plan which will guide Hawaii education over the course of the coming years, teachers and citizens should encourage DOE officials to fully embrace the sobering findings of “No Time to Lose,” the tremendous energy and wealth of ideas emerging from Gov. David Ige’s task force, and the “Schools Our Keiki Deserve” report.

The report outlines a plan that is fully in accordance with the best educational research — one that can and should be integrated into the blueprint of the document that will determine much of what happens in our schools.

Categories Hawaii, Teacher Evaluations, VAM (value-added modeling)

Karin Klein: Why Reformers Were Wrong about Teachers

By dianeravitch
September 6, 2016 //
25

Karin Klein wrote education editorials for the Los Angeles Times for years. She now writes freelance, and she wrote this sensible article for the LA Times.

So-called reformers have advocated their view that the way to improve schools is to fire “bad” teachers. The way they would identify “bad” teachers is by whether the test scores of students went up or down or stayed flat. Reformers seldom acknowledged that test scores reflect family income far more than teacher quality.

This hunt for bad teachers has proved fruitless, as scores have misidentified good and bad teachers, good teachers are demoralized by an idiotic way of evaluating their work, and there are teacher shortages now in many districts, as good teachers leave and the pipeline of new teachers has diminishing numbers.

Linda Darling-Hammond once memorably said, “You can’t fire your way to Finland.”

Karin Klein agrees.

One day, when the current era of test-based evaluation is evaluated, reformers will be held accountable for the damage they have done to teachers, students, and public education. That day will come.

Teachers need help and support to become better teachers.

There is no waiting line of great teachers searching for a job.

School districts must work with the teachers they have, making sure they are encouraged and mentored. And paid well.

Categories Accountability, Corporate Reformers, Teacher Education, Teacher Evaluations, Teacher Pay, Teacher Shortage, Teachers and Teaching

New Jersey Triples Weight of Test Scores in Teacher Evaluations

By dianeravitch
September 3, 2016 //
15

New Jersey has decided that teachers are now fully familiar with the Common Core and PARCC testing, even though most kids “fail” it, and henceforward the rise or fall of test scores on PARCC will count for 30% of teacher evaluations. Previously they had counted for only 10%.

This method has been debunked by the American Statistical Association and the American Educational Research Association. It has been in use in Colorado and in many states for five years without producing any results.

This is faith-based policy.

http://www.nj.com/education/2016/08/nj_parcc_tests_teacher_evaluations.html

The only sensible aspect of this change is that it counts only for teachers who teach the tested subjects in the tested grades. In neighboring New York and in other states, this discredited method applies to all teachers, and they are judged by the scores of students they didn’t teach in subjects they don’t teach.

In New York, an outstanding fourth grade teacher, Sheri Lederman, sued the state after receiving a low rating. The judge ruled that the rating system was “arbitrary and capricious.” For now, the rating system is in abeyance. At some point the Regrnts and Legislature will have to clarify how this ruling affects state law.

Categories New Jersey, Teacher Evaluations, VAM (value-added modeling)

Florida: Candidate for Orange County School Board Runs Against Marzano Method

By dianeravitch
August 21, 2016 //
26

This is an unusual political campaign. Matthew Fitzpatrick, an educator in Orange County, Florida, is running for a seat on the district school board on a platform opposed to the evaluation methods of Robert Marzano. Now, I have no views for or against Mr. Marzano since I am not a classroom teacher and I am not familiar with his method, but I have seen remarkable pushback on this blog from teachers. Since I too oppose the reduction of teaching to numerical measurements, I am sympathetic to his arguments.

He gives 40 reasons to oppose the Marzano method. I am posting only four of them. Read his post if you want to see the other 36.

My name is Matthew J. Fitzpatrick, and I am running for the District 7 Seat on the Orange County School Board. I am currently an Assistant Director at Orange Technical College, Westside Campus in Winter Garden. I’ve been in education for 23 years — 12 years as a teacher, and 11 years as a school and district administrator. In all my years of being involved in education, in my opinion, I have never seen a more demoralizing and destructive system than the OCPS implementation of the Marzano Teacher Evaluation system. I believe the Marzano system, more than anything else, is driving teachers out of education…and thus, OCPS has long lists of teacher vacancies. I believe this enough that I am willing to set aside my own administrative career and take a 50% pay cut in order to bring common sense back to the classroom. We must turn things around now.

Here are my first 40 Reasons to Replace the Marzano Teacher Evaluation System…splitting hairs on a system designed to split hairs on the art of teaching…

1. Dr. Marzano himself said on page 4 of his famous book, The Art and Science of Teaching, that, “It is certainly true that research provides us with guidance as to the nature of effective teaching, and yet I strongly believe that there is not (nor will there ever be) a formula for effective teaching.” If Dr. Robert J. Marzano says there is not a formula for effective instruction, who am I to argue with him? Why have we settle for a cookie-cutter approach to teaching?

2. Non-educators may not completely understand all of this “teacherese” jargon about teacher evaluations, but simply mention the name Marzano to an Orange County Public School teacher and take note of how they react…watch what happens to their face…feel the emotions of their words. Anything that causes such disdain among the very lifeblood of education–the teachers–surely is not good for education…no matter how much the sanitized research is quoted in support of it.

3. Where are the amazing results from using the “research-proven” Marzano strategies? Our District’s test scores and grades went down in many areas and schools. Why haven’t 6 years of Marzano transformed our District? If something is not delivering results, and at the same time it is driving great teachers out of the profession, we must make a data-driven decision and move in another direction…for the sake of our students and teachers.

4. Teaching should not be reduced to the numerical measurements of individual instructional strategies. Just as Mr. Keating (Robin Williams), in Dead Poets Society instructed his students to resist the armies of academics who want to reduce poetry to a passionless score that misses its true beauty and purpose, so, too, must students, parents, teachers and administrators stand against such a heartless, nitpicking view of the art of instruction. We must “Rip It Out” as an evaluation tool in our District.

Categories Accountability, Education Reform, Florida, Teacher Evaluations, Teachers and Teaching, Testing

Reader: It Takes 300 Days to Teach the Florida Standards Effectively

By dianeravitch
August 20, 2016 //
35

A reader who works for a software company explains why it is so difficult to teach the standards effectively and so unfair to judge teachers by an impossible task: It takes 300 days to teach them, but there are only 180 days in a school year. Oops!

 

 

Here is the main problem with these tests. The FLDOE has absolutely no clue on how long it takes to teach each standard effectively. So the question is, “can a teacher teach the standards in the allotted time during the year?” As an educational software company we looked at the standards that a fifth grade teacher is required to teach effectively and stopped counting when we found it would take a minimum of at least 300 school days to teach the standards to an effective level. This does not include teaching a child how to type effectively if the state required typing on the writing portion of the test. The problem is, it’s impossible for an elementary school teacher or for that matter anybody including the testing companies to teach the standards that are on the test in a school year. In order for a teacher or school to score effectively on these tests you have to hope that the students that are coming into your classroom have at least some prior knowledge of the standards.

 

 

You have to understand that these tests are not built to test your child’s learning knowledge, they are built to evaluate the schools and teachers on their effectiveness on teaching the standards. Finally, ask yourself this question… “Who benefits if the teachers and schools FAIL teaching the standards effectively?” Teachers? Schools? Children? No benefit here!… Private Charter Schools? Testing Companies? Publishers? ED Tech Companies? Lobbyists and the list goes on and on and on…..

Categories Florida, Teacher Evaluations, Testing

John Ewing: How to Attract and Retain the Teachers We Need

By dianeravitch
August 14, 2016 //
10

John Ewing, mathematician, is CEO of Math for America, an organization that supports teachers of mathematics.

In this post, he reviews some of the recent ill-considered efforts to “respect” teachers and offers advice about the minimum conditions necessary to assure that teachers have the respect, autonomy, and trust that professionals deserve.

You will also enjoy reading John Ewing’s brilliant takedown of teacher evaluation by test scores, which he called “Mathematical Intimidation.”

Categories Education Reform, Teacher Evaluations, Teachers and Teaching, VAM (value-added modeling)

Melinda Gates: We Won’t Back Down in Face of Opposition to Our Education Reforms

By dianeravitch
August 12, 2016 //
81

Melinda Gates told the National Conference of State Legislatures that the Gates Foundation has no intention of backing away from their agenda of Common Core, teacher evaluations that include test scores, charter schools, and digital learning.

No matter how controversial, no matter how much public pushback, they are determined to stay the course. For some reason, she thinks that the foundation is a “neutral broker,” when in fact it is an advocate for policies that many teachers and parents reject. She also assumes that the Gates Foundation has “the real facts,” when in fact it has a strong point of view reflecting the will of Bill & Melinda. There was no reference to evidence or research in this account of her position. Her point was that, no matter what the public or teachers may say, no matter how they damage the profession and public education, the multi-billion dollar foundation will not back down from its priorities. The only things that can stop them are informed voters and courts, such as the vote against charter schools in Nashville and the court decision in Washington State declaring that charter schools are not public schools.

The question that will be resolved over the next decade is whether the public will fight for democratic control of public schools or whether the world’s richest man can buy public education.

Melinda Gates said she and her husband, Microsoft co-founder Bill Gates, learned an important lesson from the fierce pushback against the Common Core State Standards in recent years. Not that they made the wrong bet when they poured hundreds of millions of dollars into supporting the education standards, but that such a massive initiative will not be successful unless teachers and parents believe in it.

“Community buy-in is huge,” Melinda Gates said in an interview here on Wednesday, adding that cultivating such support for big cultural shifts in education takes time. “It means that in some ways, you have to go more slowly.”

That does not mean the foundation has any plans to back off the Common Core or its other priorities, including its long-held belief that improving teacher quality is the key to transforming public education. “I would say stay the course. We’re not even close to finished,” Gates said.

The Bill & Melinda Gates Foundation has helped shape the nation’s education policies during the past decade with philanthropic donations that have supported digital learning and charter schools and helped accelerate shifts not only to the new, common academic standards, but to new teacher evaluations that incorporate student test scores.

The Obama administration shared and promoted many of the foundation’s priorities, arguing that they were necessary to push the nation’s schools forward and close yawning achievement gaps. Now that a new federal education law has returned authority over public education to the states, the foundation is following suit, seeking to become involved in the debates about the direction of public schools that are heating up in state capitals across the country.

Speaking here at a meeting of the National Conference of State Legislatures, Melinda Gates told lawmakers on Wednesday that the new federal education law, the Every Student Succeeds Act, gives them a chance to grapple with whether “we are doing everything in our power to ensure that students are truly graduating ready to go on to meaningful work or to college.”

“I want the foundation to be the neutral broker that’s able to bring up the real data of what is working and what’s not working,” Gates said in an interview afterward.

She went on to say that the foundation would continue to pursue its priorities.

“I think we know what the big elements are in education reform. It’s how do you support the things that you know work and how do you get the whole system aligned behind it,” Gates said. “I’m not telling you it’s going to be easy. There are now 50 states that have to do it, and there isn’t this federal carrot or the stick, the push or pull, to help them along.”

The agenda she described is not one that everyone considers neutral. It includes supporting the Common Core standards and developing lesson-planning materials to help teachers teach to those standards; promoting personalized learning, or digital programs meant to target students’ individual needs; and, above all, improving the quality of teachers in the nation’s classrooms, from boosting teacher preparation to rethinking on-the-job professional development.

Categories Accountability, Charter Schools, Common Core, Corporate Reformers, Education Industry, Education Reform, Gates Foundation, Bill Gates, Privatization, Resistance, Teacher Evaluations, Testing, U.S. Department of Education, VAM (value-added modeling)
« Older Entries
Newer Entries »