Archives for category: Teacher Evaluations
« Older Entries
Newer Entries »

John King Doubles Down on Importance of Standardized Tests; “Reformers” Cheer

By dianeravitch
October 12, 2016 //
25

Standardized tests produce results normed on a bell curve. The students who cluster in the bottom half of the bell curves are predominantly poor, children with disabilities, and children of color. The bell curve, by design, never closes. That is why it is fundamentally wrong to rank students, teachers, and schools by a measure that favors the most affluent.

Secretary of Education John King is releasing regulations that will punish education programs if their graduates teach students whose scores are low. “Reformers” are supposed to be aware of the power of incentives, but not Secretary King. He thinks he can scare education programs to focus more on raising test scores. More likely is that teachers will get the message to avoid teaching in schools that enroll students who are impoverished, and that their preparation programs will encourage them to steer clear of the neediest children.

This is the report that appeared this morning in politico education (http://go.politicoemail.com/?qs=70112e1be3cf7964cb8f83700178acc6d2452a022573f96620522e9d5cbe9274):

TEACHER PREP RULES OUT TODAY: The Obama administration unveiled its long-delayed final regulations governing teacher preparation programs today. The rule preserves much of the administration’s original proposal from 2014, and requires states to develop a rating system for teacher-preparation programs.

– The rule will also eventually punish low-performing programs by cutting off their access to federal TEACH grants that help students pay for teacher training.

– The final rule retains a particularly-controversial component, which holds teacher-preparation programs accountable, in part, for how their graduates perform as teachers, based upon their students’ academic success. However, states will have flexibility in determining how to measure student learning.

– Randi Weingarten, president of the American Federation of Teachers, sharply criticized the regulations, saying in a statement that although the department “has made minor tweaks, the flawed framework remains the same.”

– Weingarten said it was “ludicrous to propose evaluating teacher preparation programs based on the performance of the students taught by a program’s graduates.” And she said the rules ultimately punish teacher prep programs that send graduates into the highest-need schools.

– Chris Minnich, executive director of the Council of Chief State School Officers, said in a statement that the group was “pleased the department listened to feedback and made these regulations stronger.”

– Kate Walsh, president of the National Council on Teacher Quality, said she’s impressed with how much the department kept with its original intent, “which was to insert far greater accountability for program quality.” She added that the effectiveness of the rule “very much depends on states doing their bit to hold programs accountable for quality.”

– “I told people they would never see the light of day,” Walsh said of the rules. “I’m happy to be wrong.”

– Education Secretary John B. King Jr. will be speaking about teacher preparation at the University of Southern California’s Rossier School of Education today. Watch the USC event live, starting at 1 p.m. ET, here.

– Read the regulations here

Click to access teacher-prep-final-regs.pdf

Categories Accountability, Corporate Reformers, Education Industry, John King, Teacher Education, Teacher Evaluations, Testing

Dilbert Explains How VAM Really Works

By dianeravitch
October 9, 2016 //
11

It takes a comedian or a cartoonist to explain the nutty world of education reform.

Check out this great cartoon by Dilbert, giving a fast explanation of the idiocy of VAM.

Enjoy!

PS: Thanks for KrazyTA for sending me the cartoon and also giving the correct link!

Categories Humor, Teacher Evaluations, VAM (value-added modeling)

Cathy O’Neil: The Fallacy of VAM as “One of Multiple Measures”

By dianeravitch
October 8, 2016 //
28

Cathy O’Neil has written s new book called “Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy.” I haven’t read it yet, but I will.

In this article, she explains that VAM is a failure and a fraud. The VAM fanatics in the federal Department of Education and state officials could not admit they were wrong, could not admit that Bill Gates had suckered the nation’s education leaders into buying his goofy data-based evaluation mania, and could not abandon the stupidity they inflicted on the nation’s teachers and schools. So they say now that VAM will be one of many measures. But why include an invalid measure at all?

As she is out on book tour, people ask questions and the most common is that VAM is only one of multiple measures.

She writes:

“Here’s an example of an argument I’ve seen consistently when it comes to the defense of the teacher value-added model (VAM) scores, and sometimes the recidivism risk scores as well. Namely, that the teacher’s VAM scores were “one of many considerations” taken to establish an overall teacher’s score. The use of something that is unfair is less unfair, in other words, if you also use other things which balance it out and are fair.

“If you don’t know what a VAM is, or what my critique about it is, take a look at this post, or read my book. The very short version is that it’s little better than a random number generator.

“The obvious irony of the “one of many” argument is, besides the mathematical one I will make below, that the VAM was supposed to actually have a real effect on teachers assessments, and that effect was meant to be valuable and objective. So any argument about it which basically implies that it’s okay to use it because it has very little power seems odd and self-defeating.

“Sometimes it’s true that a single inconsistent or badly conceived ingredient in an overall score is diluted by the other stronger and fairer assessment constituents. But I’d argue that this is not the case for how teachers’ VAM scores work in their overall teacher evaluations.

“Here’s what I learned by researching and talking to people who build teacher scores. That most of the other things they use – primarily scores derived from categorical evaluations by principals, teachers, and outsider observers – have very little variance. Almost all teachers are considered “acceptable” or “excellent” by those measurements, so they all turn into the same number or numbers when scored. That’s not a lot to work with, if the bottom 60% of teachers have essentially the same score, and you’re trying to locate the worst 2% of teachers.

“The VAM was brought in precisely to introduce variance to the overall mix. You introduce numeric VAM scores so that there’s more “spread” between teachers, so you can rank them and you’ll be sure to get teachers at the bottom.

“But if those VAM scores are actually meaningless, or at least extremely noisy, then what you have is “spread” without accuracy. And it doesn’t help to mix in the other scores.”

This is a book I want to read. Bill Gates should read it too. Send it to him and John King too. Would they read it? Not likely.

Categories Accountability, Corporate Reform, Data, Education Industry, Failure, Fraud, Teacher Evaluations, VAM (value-added modeling)

This is a Classic: The American Statistical Association Statement on Value-Added Measurement

By dianeravitch
October 5, 2016 //
21

When this statement first appeared in 2014, I said at the time that it should be on the bulletin board of every public school.

The American Statistical Association explains here why the evaluations of individual teachers should not be based on their students’ test scores.

Here is an excerpt. Read the whole statement, which is only 8 pages long:

It is unknown how full implementation of an accountability system incorporating test-based indicators, such as those derived from VAMs, will affect the actions and dispositions of teachers, principals and other educators. Perceptions of transparency, fairness and credibility will be crucial in determining the degree of success of the system as a whole in achieving its goals of improving the quality of teaching. Given the unpredictability of such complex interacting forces, it is difficult to anticipate how the education system as a whole will be affected and how the educator labor market will respond. We know from experience with other quality improvement undertakings that changes in evaluation strategy have unintended consequences. A decision to use VAMs for teacher evaluations might change the way the tests are viewed and lead to changes in the school environment. For example, more classroom time might be spent on test preparation and on specific content from the test at the exclusion of content that may lead to better long-term learning gains or motivation for students. Certain schools may be hard to staff if there is a perception that it is harder for teachers to achieve good VAM scores when working in them. Overreliance on VAM scores may foster a competitive environment, discouraging collaboration and efforts to improve the educational system as a whole.

Research on VAMs has been fairly consistent that aspects of educational effectiveness that are measurable and within teacher control represent a small part of the total variation in student test scores or growth; most estimates in the literature attribute between 1% and 14% of the total variability to teachers. This is not saying that teachers have little effect on students, but that variation among teachers accounts for a small part of the variation in scores. The majority of the variation in test scores is attributable to factors outside of the teacher’s control such as student and family background, poverty, curriculum, and unmeasured influences.

The VAM scores themselves have large standard errors, even when calculated using several years of data. These large standard errors make rankings unstable, even under the best scenarios for modeling. Combining VAMs across multiple years decreases the standard error of VAM scores. Multiple years of data, however, do not help problems caused when a model systematically undervalues teachers who work in specific contexts or with specific types of students, since that systematic undervaluation would be present in every year of data.

Despite the warning from ASA, which has no special interest and does not represent teachers or public school administrators, many states continue to use this method (called VAM, or value-added measurement or value-added modeling).

States were coerced into adopting this unproven method by the U.S. Department of Education, which said that states had to adopt it if they wanted to be eligible to compete for nearly $5 billion in federal funds in 2009, as every state was undergoing a budget crisis caused by the economic meltdown of fall 2008.

Many states adopted it, and it has not had positive effects in any state.

In Colorado and New York, among others, VAM scores count for as much as 50% of teachers’ evaluation.

A state court in New York ruled this method “arbitrary and capricious” when challenged by fourth grade teacher Sheri Lederman and her lawyer-husband Bruce Lederman.

Some states assign VAM scores to teachers based on students they never taught in subjects they don’t teach.

This is an example of federal and state policy that has no basis in evidence and that has harmed the lives of many teachers. It very likely has caused teachers to leave the profession and contributed to teacher shortages.

Categories Race to the Top, Teacher Evaluations, VAM (value-added modeling)

Should Teachers Be Evaluated by Test Scores? Does One Teacher Affect Your Lifetime Earnings?

By dianeravitch
September 23, 2016 //
42

This post, with an anonymous author, reviews the research on value-added measurement, with frequent references to those who claim that the rise or fall of test scores is the best way to judge teacher quality.

 

The basic question he or she addresses is whether the actions of your kindergarten teacher or your third-grade teacher can affect your lifetime earnings, as Raj Chetty and his team asserted in a study a few years back.

 

The author goes into a lengthy back-and-forth about whether such claims make sense.

 

But the one essential fact that his post is missing is that 70% of teachers do not teach tested subjects. A district or school can evaluate teachers with VAM only when there are enough years of test scores to document the effects of the teacher over several years. Teachers of subjects other than reading and mathematics in grades 3-8 will never get VAM ratings.

 

But many states have solved this problem by assigning VAM ratings to the 70%. Their ratings are based on the scores of students they never met in subjects they never taught. This is called an “attributed rating.”

 

That makes sense, said no one ever.

 

That may be why Hawaii and Oklahoma have dropped VAM. It is expensive and gives false positives and false negatives. Expect more states to join these two states.

Categories Teacher Evaluations, VAM (value-added modeling)

Hawaii: Sorry, But the Nutty Teacher Evaluations Are Still Imposed 

By dianeravitch
September 10, 2016 //
22

Earlier today, I posted a piece by Andy Jones, a high school language arts teacher in Hawaii. In the introduction, I mistakenly said that Hawaii had eliminated the teacher evaluation system that was created in response to winning Race to the Top funding. I wrote: “This past spring, Hawaii dropped the test-based teacher evaluation that Race to the Top had forced on the state as a condition of winning RTTT funds. The money was all gone, and so was this bad idea.” That was not completely accurate.

Andy Jones sent this note to correct me:

“I so hate to ask, but…the HSTA secretary, who along with everyone else here is a great admirer of your work, just asked me if I could mention to you that your intro to my article is somewhat inaccurate, in that the teacher evaluation system (EES) is still in place. We were merely successful in eliminating test scores from the evaluation. We are, however, almost equally angry at present, because in the months since that announcement was made, they have insisted on maintaining the excoriated SLO component (Student Learning Outcomes) – the mindbogglingly ridiculous quasi-action-research template we have to go through in our evaluations in which we choose learning objectives, administer pre-assessments, make predictions about student learning, collect evidence of student growth, etc., and are only rated “distinguished” if 90% of our predictions are accurate. This is now worth 50% of our evaluation, along with the standard Danielson-based observation cycle and Core Professionalism (basically a portfolio on Danielson Domain 4) comprising the other 50%.

“HSTA secretary Amy Perruso wanted me to post you on this, because she’s afraid that HSTA will now be bombarded with emails and telephone calls asking “How did you get rid of the teacher evaluations?” – which, unfortunately we have not and which we’re going to be attempting very aggressively this year to get rid of.

– Andy”

Categories Education Reform, Hawaii, Race to the Top, Teacher Evaluations

Andy Jones: Hawaii’s Education Future Is Looking Brighter

By dianeravitch
September 10, 2016 //
1

Andy Jones is a high school language arts teacher in Hawaii. The public school teachers in Hawaii have been energetic in stopping the worst extremes of the Race to the Top grant that the state received. This past spring, Hawaii dropped the test-based teacher evaluation that Race to the Top had forced on the state as a condition of winning RTTT funds. The money was all gone, and so was this bad idea.

Andy knows the history of American education, the cycles of criticism that follow one another, and he believes that the new federal law will give the state the ability to chart a better course than the one imposed by the Bush-Obama agenda of test and punish. I regret to say that his article is behind a pay wall, but I will share with you what he shared with me. In this time of doom-and-gloom, it is great to hear an optimistic forecast!

Andy writes:


Alarm, overhaul, stagnation, denial, recognition. Repeat ad infinitum.

Students of educational history will recognize in these five words the cycle that has defined American school culture for decades.

Thirty-three years ago, the “A Nation at Risk” report rang the alarm of educational decline. Its clarion call for improved public education resounded for the next generation and led eventually to initiatives such as No Child Left Behind and Race to the Top – programs that inaugurated sweeping, possibly irreversible changes to schools and school communities across the country.

A consensus has now emerged that these changes have led to dismal failure — a consensus signaled by the Every Student Succeeds Act (ESSA), which emphatically seeks to reverse the damage done, in part by giving states back the freedom to define and enact their own vision of 21st-century education.

On the heels of ESSA and the widespread discussion it has initiated, the National Conference of State Legislatures (NCSL) has released a report that may prove as decisive as “A Nation at Risk.”

The title — “No Time to Lose: How to Build a World-Class Education System State by State” — is misleading in that it seems to announce a mere repeat of the alarmist tone of the “Nation” report, perhaps to be followed by a new round of dubious policy suggestions from non-educators.

However, in what must come as a welcome shock to educators accustomed to routine governmental denial of policy failure, “No Time to Lose” fully acknowledges the mistakes of the past 15 years and seconds the sustained criticisms of prominent researchers such as Diane Ravitch and Pasi Sahlberg. These and many others have analyzed the extensive Organisation for Economic Co-operation and Development reports on international education and have concluded that the misguided “reforms” of the past years have had an overwhelmingly negative impact on American schools, leading to ever further decline internationally.

They have also highlighted an additional, sinister aspect of these “reforms,” which have involved the gradual removal of educational decisions from the purview of teachers and educators and the corresponding enrichment of educa- tional corporations profiting from the proliferation of mediocre materials and programs that schools are forced to use.

We are fortunate to be living in a state led by a governor who recognizes what is at stake and who has created a robust task force that is working to establish grassroots consensus as to what is best for Hawaii schools and the students they serve.

We are also fortunate to have an increasingly dynamic teachers union that has sponsored a teacher- written report, “Schools Our Keiki Deserve,” which echoes the advice of our top educational researchers as well as the urgent tone of “No Time to Lose.”

Hawaii Department of Education (DOE) officials have shown signs recently that they are beginning to veer away from the pattern of denial that for years has characterized state and district education departments across the country. They have, for instance, conceded the unhealthy aspects of standardized testing, and they have also begun to embrace the idea of whole-child education as practiced in the world’s top-performing school systems.

As the DOE continues revising the Strategic Plan which will guide Hawaii education over the course of the coming years, teachers and citizens should encourage DOE officials to fully embrace the sobering findings of “No Time to Lose,” the tremendous energy and wealth of ideas emerging from Gov. David Ige’s task force, and the “Schools Our Keiki Deserve” report.

The report outlines a plan that is fully in accordance with the best educational research — one that can and should be integrated into the blueprint of the document that will determine much of what happens in our schools.

Categories Hawaii, Teacher Evaluations, VAM (value-added modeling)

Karin Klein: Why Reformers Were Wrong about Teachers

By dianeravitch
September 6, 2016 //
25

Karin Klein wrote education editorials for the Los Angeles Times for years. She now writes freelance, and she wrote this sensible article for the LA Times.

So-called reformers have advocated their view that the way to improve schools is to fire “bad” teachers. The way they would identify “bad” teachers is by whether the test scores of students went up or down or stayed flat. Reformers seldom acknowledged that test scores reflect family income far more than teacher quality.

This hunt for bad teachers has proved fruitless, as scores have misidentified good and bad teachers, good teachers are demoralized by an idiotic way of evaluating their work, and there are teacher shortages now in many districts, as good teachers leave and the pipeline of new teachers has diminishing numbers.

Linda Darling-Hammond once memorably said, “You can’t fire your way to Finland.”

Karin Klein agrees.

One day, when the current era of test-based evaluation is evaluated, reformers will be held accountable for the damage they have done to teachers, students, and public education. That day will come.

Teachers need help and support to become better teachers.

There is no waiting line of great teachers searching for a job.

School districts must work with the teachers they have, making sure they are encouraged and mentored. And paid well.

Categories Accountability, Corporate Reformers, Teacher Education, Teacher Evaluations, Teacher Pay, Teacher Shortage, Teachers and Teaching

New Jersey Triples Weight of Test Scores in Teacher Evaluations

By dianeravitch
September 3, 2016 //
15

New Jersey has decided that teachers are now fully familiar with the Common Core and PARCC testing, even though most kids “fail” it, and henceforward the rise or fall of test scores on PARCC will count for 30% of teacher evaluations. Previously they had counted for only 10%.

This method has been debunked by the American Statistical Association and the American Educational Research Association. It has been in use in Colorado and in many states for five years without producing any results.

This is faith-based policy.

http://www.nj.com/education/2016/08/nj_parcc_tests_teacher_evaluations.html

The only sensible aspect of this change is that it counts only for teachers who teach the tested subjects in the tested grades. In neighboring New York and in other states, this discredited method applies to all teachers, and they are judged by the scores of students they didn’t teach in subjects they don’t teach.

In New York, an outstanding fourth grade teacher, Sheri Lederman, sued the state after receiving a low rating. The judge ruled that the rating system was “arbitrary and capricious.” For now, the rating system is in abeyance. At some point the Regrnts and Legislature will have to clarify how this ruling affects state law.

Categories New Jersey, Teacher Evaluations, VAM (value-added modeling)

Florida: Candidate for Orange County School Board Runs Against Marzano Method

By dianeravitch
August 21, 2016 //
26

This is an unusual political campaign. Matthew Fitzpatrick, an educator in Orange County, Florida, is running for a seat on the district school board on a platform opposed to the evaluation methods of Robert Marzano. Now, I have no views for or against Mr. Marzano since I am not a classroom teacher and I am not familiar with his method, but I have seen remarkable pushback on this blog from teachers. Since I too oppose the reduction of teaching to numerical measurements, I am sympathetic to his arguments.

He gives 40 reasons to oppose the Marzano method. I am posting only four of them. Read his post if you want to see the other 36.

My name is Matthew J. Fitzpatrick, and I am running for the District 7 Seat on the Orange County School Board. I am currently an Assistant Director at Orange Technical College, Westside Campus in Winter Garden. I’ve been in education for 23 years — 12 years as a teacher, and 11 years as a school and district administrator. In all my years of being involved in education, in my opinion, I have never seen a more demoralizing and destructive system than the OCPS implementation of the Marzano Teacher Evaluation system. I believe the Marzano system, more than anything else, is driving teachers out of education…and thus, OCPS has long lists of teacher vacancies. I believe this enough that I am willing to set aside my own administrative career and take a 50% pay cut in order to bring common sense back to the classroom. We must turn things around now.

Here are my first 40 Reasons to Replace the Marzano Teacher Evaluation System…splitting hairs on a system designed to split hairs on the art of teaching…

1. Dr. Marzano himself said on page 4 of his famous book, The Art and Science of Teaching, that, “It is certainly true that research provides us with guidance as to the nature of effective teaching, and yet I strongly believe that there is not (nor will there ever be) a formula for effective teaching.” If Dr. Robert J. Marzano says there is not a formula for effective instruction, who am I to argue with him? Why have we settle for a cookie-cutter approach to teaching?

2. Non-educators may not completely understand all of this “teacherese” jargon about teacher evaluations, but simply mention the name Marzano to an Orange County Public School teacher and take note of how they react…watch what happens to their face…feel the emotions of their words. Anything that causes such disdain among the very lifeblood of education–the teachers–surely is not good for education…no matter how much the sanitized research is quoted in support of it.

3. Where are the amazing results from using the “research-proven” Marzano strategies? Our District’s test scores and grades went down in many areas and schools. Why haven’t 6 years of Marzano transformed our District? If something is not delivering results, and at the same time it is driving great teachers out of the profession, we must make a data-driven decision and move in another direction…for the sake of our students and teachers.

4. Teaching should not be reduced to the numerical measurements of individual instructional strategies. Just as Mr. Keating (Robin Williams), in Dead Poets Society instructed his students to resist the armies of academics who want to reduce poetry to a passionless score that misses its true beauty and purpose, so, too, must students, parents, teachers and administrators stand against such a heartless, nitpicking view of the art of instruction. We must “Rip It Out” as an evaluation tool in our District.

Categories Accountability, Education Reform, Florida, Teacher Evaluations, Teachers and Teaching, Testing
« Older Entries
Newer Entries »