Archives for category: Teacher Evaluations

Bob Shepherd writes on the absurd demands now placed on teachers and principals by politicians, who expect to see higher test scores every year. Step back and you realize that the politicians, the policy wonks, the economists, and the ideologues are ruining education, not improving it. They are doing their best to demoralize professional educators. What are they thinking? Are they thinking? Or is it just their love of disruption, let loose on children, families, communities, and educators?

Bon Shepherd writes:

OK, you are sitting in your year-end evaluation session, and you’ve heard from every other teacher in your school that his or her scores were a full level lower this year than last, and so you know that the central office has leaned on the principal to give fewer exemplary ratings even though your school actually doesn’t have a problem with its test scores and people are doing what they did last year but a bit better, of course, because one grows each year as a teacher–one refines what one did before, and one never stops learning.

But you know that this ritual doesn’t have anything, really, to do with improvement. It has to do with everyone, all along the line, covering his or her tushy and playing the game and doing exactly what he or she is told. And, at any rate, everyone knows that the tests are not particularly valid and that’s not really the issue at your school because, the test scores are pretty good because this is a suburban school with affluent parents, and the kids always, year after year, do quite well.

So whether the kids are learning isn’t really the issue. The issue is that by some sort of magic formula, each cohort of kids is supposed to perform better than the last–significantly better–on the tests, though they come into your classes in exactly the same shape they’ve always come into them in because, you know, they are kids and they are just learning and teaching ISN’T magic. It’s a lot of hard work. It’s magical, sometimes, of course, but its’ not magic. There’s no magic formula.

So, the stuff you’ve been told to do in your “trainings” (“Bark. Roll over. Sit. Good Boy”) is pretty transparently teaching-to-the-test because that’s the only way the insane demand that each cohort will be magically superior to the last as measured by these tests can be met, but you feel in your heart of hearts that doing that would be JUST WRONG–it would short-change your students to start teaching InstaWriting-for-the-Test, Grade 5, instead of, say, teaching writing. And despite all the demeaning crap you are subjected to, you still give a damn.

And you sit there and you actually feel sorry for this principal because she, too, is squirming like a fly in treacle in the muck that is Education Deform, and she knows she has fantastic teachers who knock it out of the park year after year, but her life has become a living hell of accountability reports and data chats to the point that she doesn’t have time for anything else anymore (she has said this many times), and now she has to sit there and tell her amazing veteran teachers who have worked so hard all these years and who care so much and give so much and are so learned and caring that they are just satisfactory, and she feels like hell doing this and is wondering when she can retire.

And the fact that you BOTH know this hangs there in the room–the big, ugly, unspoken thing. And the politicians and the plutocrats and the policy wonks at the Thomas B. Fordham Institute and the Secretary of the Department for the Standardization of US Education, formerly the USDE, and the Vichy education guru collaborators with these people barrel ahead, like so many drunks in a car plowing through a crowd of pedestrians.

New Mexico recently released teacher ratings, 50% based on standardized test scores. The teachers are hopping mad, because they know that the evaluations do not truly measure their quality, and the tests are not good measures of what students know and can do.

In Taos, teachers burned their evaluation reports. Teachers in Albuquerque also burned their evaluations as a sign of protest.

During the Vietnam war, anti-war protestors burned their draft cards. Feminists burned their bras in protest at the Miss America contest in 1968.

This is a venerable protest activity against injustice.

Audrey Amrein-Beardsley, one of our nation’s pre-eminent experts on value-added assessment, here reviews a TED-X talk by Tennessee Commissioner of Education Kevin Huffman, boasting of the tremendous growth in test scores as a result of his policies. Beardsley points out the curious fact that Tennessee started using VAM in the 1990s with little to show for it. But, there were those Tennessee NAEP scores, proof positive, according to both Huffman and Se rotary of Education Arne Duncan that Race to the Top–or Huffman’s personal presence–was creating strong results. Nd in the end, results (test scores) are what matter most, right?

But what about those NAEP results that Huffman and Duncan tout?

Beardsley writes:

“While [William] Sanders (the TVAAS developer who first convinced the state legislature to adopt his model for high-stakes accountability purposes in the 1990s) and others (including U.S. Secretary of Education Arne Duncan) also claimed that Tennessee’s use of accountability instruments caused Tennessee’s NAEP gains (besides the fact that the purported gains were over two decades delayed), others have since spoiled the celebration because 1) the results also demonstrated an expanding achievement gap in Tennessee; 2) the state’s lowest socioeconomic students continue to perform poorly, despite Huffman’s claims; 3) Tennessee didn’t make gains significantly different than many other states; and 4) other states with similar accountability instruments and policies (e.g., Colorado, Louisiana) did not make similar gains, while states without such instruments and policies (e.g., Kentucky, Iowa, Washington) did. I should add that Kentucky’s achievement gap is also narrowing and their lowest socioeconomic students have made significant gains. This is important to note as Huffman repeatedly compares his state to theirs.”

Read the post. It is a very good demonstration of how data get used and misused for political purposes.

Thus far, the concept of VAM–or value-added measurement–has an unbroken record of failure. Wherever it has been tried, it has proven to be inaccurate and unstable. Teacher and student records are erroneous. Teachers are judged based on students they never taught. VAM demoralizes teachers, who understand they are being judged for factors over which they have little or no control.

The major perpetrators of this great fraud are Bill Gates, who bet hundreds of millions of dollars on the proposition that test scores could be a major factor in identifying bad teachers and firing them, and Arne Duncan, who required states to use VAM if they wanted to be eligible to get a share of his $4.35 billion Race to the Top fund.

Yet a third perpetrator was Jeb Bush, whose love affair with data is unbounded. Bush went from state to state selling “the Florida miracle,” which supposedly proved that testing and accountability were the keys to solving America’s educational problems.

One of Jeb’s acolytes was Hannah Skandera, who was chosen as Secretary of Education in Néw Mexico but was never confirmed because of her lack of classroom credentials. As Secretary-designate, she sought to import the Florida model of testing and accountability.

When the state released its new teacher evaluation ratings, teachers and students showed up at the Albuquerque school board meeting to complain about errors. Teachers talked about missing and incomplete data. One student said he was part of a team that placed first in the state in civics, yet he failed his end-of-course government exam.

“James Phillips teaches calculus to Advanced Placement students at Albuquerque High School. He described how the previous week had seen him publicly praised by board member Marty Esquivel, who called him the best math teacher in New Mexico. Just days later, Phillips was notified that the PED had also ranked him “minimally effective.”

“Wendy Simms-Small, a parent of three APS students who’d helped organize the day’s rally, said she started getting active after hearing rumors that hundreds of teachers were planning on leaving the school system.

“I got curious and wanted to find out why,” she said. “As a member of this community over many years, I have never seen the demoralization of professional individuals like this ever before.” She said the pressure of testing had also taken a toll on her kids.

“Private corporations reap great rewards when school systems implement standardized testing,” said Simms-Small, “so it’s my belief that they’re motivated financially to turn our children into pawns for profit.”

At some point, the data-obsessed federal and state policy makers will have to concede that they were wrong or face a massive parent-teacher rebellion. They are literally destroying the nation’s schools with their mad ideas. It is time for a revival of common sense or a public discussion of the true meaning of education.

This is one of the funniest YouTube videos I have ever seen (excluding a few about dogs and cats).

I promise you, you will not be disappointed. Watch it and forward it to your superintendent or your school board. Remember how I always say that VAM is Junk Science. Here is one of the world’s leading assessment experts, and his message mocks VAM as just plain Junk.

It features the great assessment expert W. James Popham as pitchman for a line of products guaranteed to raise your students’ test scores. Some of them are chewable, some are drinkable and last five hours (he warns that if the effect lasts for longer than five hours, call your physician at once).

Please watch this. You will never think about value-added assessments or rubrics the same way again.

Want to know more about Popham? Read this author bio.He is one funny guy.

Thanks to Arne Duncan, almost every state now has an elaborate teacher evaluation plan. There is no evidence that the pans identify teachers correctly, but they are widespread because Duncan believes and he is Secretary of Education, with more certainty than any of his predecessors.

What hath Arne wrought? Here is an account I hope he reads. It shows what a mess he has made in thousands of schools.

I”ve been trying to find the right place to share what I’ve written about the ridiculous evaluation process that occurs in Palm Beach County, FL. Friends who have read it asked that I share it with you, Diane.

My teacher evaluation rant

I’m about to write out the long, stupidly involved story of the truth behind teacher evaluations, specifically at my school, but likely not too different than everywhere.

Five (or more) times in the school year my principal does classroom observations. There are three different kinds: 1 formal, forty minute lesson observation; 2 5-15 minute informal; 2 30 second-2 minute walkthroughs. She evaluates me using the Robert Marzano Menu of Design Questions There are about 60 specific behaviors within 4 different domains, each with 3-7 components, that she is looking for during those observations.

Each of the 60 behaviors is then graded on a scale of: not using, beginning, developing, applying, and innovating. The evaluator, after marking off the components then decides how to grade the behavior. Comments, if appropriate are also added (as in: All of the students were actively engaged in the lesson) At the end of each observation I get an email directing me to approve the observation.

At the end of the year, the grades for each behavior are calculated to determine if the teacher is: Highly Effective (3.2 – 4.0), Effective (2.1 – 3.1), Developing (1.2 – 2.0) or Unsatisfactory (1.0 – 1.1). Last year I was deemed Highly Effective based on my observations. I didn’t really look at the details because I was overly pleased with the results.

I should note that for my first two years teaching this was a new evaluation system. Our district decided that during the learning curve process ALL teachers would be given the same grade/evaluation level of Effective, so the observations were a tool for us to begin to look at where we could stand to improve and what we were already doing well. The fact that I was evaluated as Highly Effective had no bearing on anything, since ALL teachers were graded as Effective. Also, last year only 2 of the 4 domains were observed. This year only 3 were observed.

The added domain is for our own personal professional development. I mention this because we were instructed at the beginning of the year that ALL teachers had to have the same personal professional development goal: to improve student success through implementation of the Marzano Techniques of Teaching (see a trend?). I didn’t want this to be my goal, but I didn’t have a choice (don’t get me started). (Last year my personal goal was to improve my ELL students’ oral language assessments by 50% – I reached that goal and then some).

This year’s evaluation has come back and I am now graded as Effective with an overall score of 3.0. I was wondering how I dropped from Highly Effective to Effective, so I started looking more closely at the numbers. Here is what I saw:

I was marked as Innovating (4.0) for 12/31 behaviors
I was marked as Applying (3.0)for 19/31 behaviors.
I had no lower marks than that.

Now in my world of calculating scores, I would multiply 4 x 12 = 48 and 3 x19=57, then add them together 48 + 57 = 105, then divide by 31 which equals 3.39. 3.39 is Highly Effective, but I was graded as 3.0 – Effective. Hmm. I called my union rep and she was not sure how that could be. She also, for what it’s worth, had a similar score drop. She remembered that there was some ridiculous way to calculate the previous two years, and thought maybe they are doing the same thing this year. Regardless, neither of us knew how our scores were calculated, so we knew we would have to ask higher ups.

I asked my principal. She wasn’t sure how it is done, either. She suggested that we both call/email the woman at the district who is in charge. So I did. This is what I learned: If 50% or more of your marks are Highly Effective, then you are Highly Effective; if 50% or more are effective than you are effective, and so on.

Then I began to wonder, as has my union rep, how is it that I was more effective last year than I am this year? What am I NOT doing now that I did then? It turns out that I am, in fact doing the same things. I was marked as doing the same exact components of behaviors this year as I was for last year. The difference is, last year I was rated as innovating more times. So, for example let’s say Behavior A has 6 components. Last year when I was checked off as meeting all 6 I was deemed innovating. This year those 6 components checked off are only earning me applying. WHY? HOW? Fortunately, I am friends with a number of people who have some real answers.

The answer isn’t pretty, but it’s been corroborated by more than one source. Here we go…

The principals and assistant principals were told that they were *giving out too many innovatings and that they needed to mark innovating less often*. In other words, the evaluation that is supposed to determine our level of teaching, which in turn determines our merit pay (no, we don’t really get merit pay. we’re supposed to, but that’s a whole different – let’s lie to the people of Florida – nightmare) is being manipulated by the powers that be in an effort to…I don’t know…make it seem like teachers aren’t as good as we are. So they can pay us less and blame us more. The powers that be are doing to the teachers what the high stakes testers are doing to our students: creating a system that is skewed to failure (or mediocrity).

I am outraged. Mostly I am outraged that Principals and ASsistant principals, who know this is wrong and are being asked to downgrade their own teachers, are going along with it and not fighting back. The people who told me are in those leadership positions. They know it’s wrong. But they pooh-pooh saying things like, “Everyone knows those evaluations aren’t right, so what does it matter? *I* know who’s great and doing a great job, so the evaluation doesn’t really matter between you and me and the kids. ”

And I almost buy it. Except for this: my evaluation is public recod. Any parent can go to the district and access my evaluation score. Most parents won’t, it is true. Only parents who have a beef with a teacher would do that, as it so happens. But that’s when the difference between effective and highly effective DOES matter. I AM highly effective. I know it. My administrators know it. My students know it. But when a parent who is already certain I am picking on his/her kid or that I don’t know how to teach his/her kid goes to access my records, they see I am Effective, not Highly Effective. It’s fuel for their fodder, which I do not like.

And if I go to work in another district, all the hiring people will see is Effective. I don’t like it. NOt one little bit.

Gary Rubinstein writes in this post about Michael Johnston and his long association with him.

Today Johnston is known in Colorado as the state senator who wrote the most punitive, anti-teacher law in the nation. At present, Harvard students are protesting the invitation to Johnston to speak at commencement

Gary knew him from Teach for America. He describes a young man who understood and cared about his students, who saw the obstacles they confronted, and who appreciated the hard work of veteran teachers.

But something happened to Michael Johnston between 2002 and 2010. The man Gary knew turned into an accountability hawk. He became a harsh critic of teachers.

For a time he was leading the test-and-punish parade, but the parade seems to be in disarray. It is no longer the leading edge but the rearguard.

Michael Johnston was invited to be the commencement speaker at the Harvard Graduate School of Education for 2014, but some students objected and called on the school to withdraw the invitation. That’s not likely to happen, nor should it. The students and graduates should have a chance to debate the issues, to debate the value of the Rhee–Duncan-Spellings style that has long been favored by the Harvard Graduate School of Education. Now is a good time to review the research on value-added measurement. Now is a good opportunity to ask SenatorJohnston what happened to quash his youthful wisdom.

John Kuhn is the superintendent of the small Perrin-Whitt Independent School District in rural Texas. He is an eloquent speaker and supporter of public education. He has spoken at national events and recently published two new books. He knows that the schoolssuffernot only fro budget cuts but from Washington’s wildly unrealistic expectations. He knows it would be nice if every student were bound for college but he knows it is unrealistic and turns success into failure.

This is a wonderful interview with the Texas Tribune. You will enjoy reading it.

This is the last Q&A:

“Trib+Edu: How has your life been different since 2011?

Kuhn: Not a whole lot different in terms of my day-to-day life. I still basically do what I’ve always done for a living and that is work in a rural public school and try to serve my community to the very best of my ability. I’ve been invited to give some speeches here and there and I’ve written a couple of books … I think speaking out like I did put me in a situation to where I’ve been educated in the political reality that affects local schools.

Previously, I just kind of accepted whatever rolled down from Washington, D.C., and whatever rolled down from Austin. I kind of thought the role of a teacher and educator was just to live with dumb policies. And I don’t think that anymore. I think now that I have a moral obligation to speak up and say, “Hey, this policy is dumb. It doesn’t work and this is what we’re seeing on the frontlines.”

I’m a fan of public education. I grew up in a little, rural Texas town where the public school was the center of what we did in town. There was no mayor’s office. It’s an unincorporated town and the school was the heart of the community. And I think, politically, we’ve kind of forgotten how important public schooling is in Texas.”

The studies of value-added measurement keep on coming, and the findings usually show what an utterly absurd idea it to think that teacher quality can be judged by student test scores. In a just world, Arne Duncan would be held accountable for the stupid and harmful theories he has imposed on the nation’s public schools. The U.S. Department of Education has become a malignant force in American education. I cannot think of any time in our nation’s history when public schools and teachers were literally endangered by the mandates coming from Washington, D.C., where the leadership is wholly ignorant of federalism.

This story in Education Week summarizes the latest batch of studies of VAM. some researchers, having made this their area of specialization, continue to prod in hopes of good news.

But look at this:

“In a study that appears in the current issue of the American Educational Research Journal, Noelle A. Paufler and Audrey Amrein-Beardsley, a doctoral candidate and an associate professor at Arizona State University, respectively, conclude that elementary school students are not randomly distributed into classrooms. That finding is significant because random distribution of students is a technical assumption that underlies some value-added models.

“Even when value-added models do account for nonrandom classroom assignment, they typically fail to consider behavior, personality, and other factors that profoundly influenced the classroom-assignment decisions of the 378 Arizona principals surveyed. That, too, can bias value-added results.

“Perhaps most provocative of all are the preliminary results of a study that uses value-added modeling to assess teacher effects on a trait they could not plausibly change, namely, their students’ heights. The results of that study, led by Marianne P. Bitler, an economics professor at the University of California, Irvine, have been presented at multiple academic conferences this year.
The authors found that teachers’ one-year “effects” on student height were nearly as large as their effects upon reading and math. The researchers did not find any correlation between the “value” that teachers “added” to height and the value they added to reading and math. In addition, unlike the reading and math results, which demonstrated some consistency from one year to the next, the height outcomes were not stable over time. The authors suggested that the different properties of the two models offered “some comfort.” Nevertheless, they advised caution.”

So, let’s get this right: teachers’ effects on students’ height were nearly as large as their effect on reading and math.

Perhaps Arne can just arrange to have all teachers fired (except for TFA), close every school (except “no-excuses” charter schools), and turnaround the whole country.

Valerie Strauss notes the growing number of studies that debunk the value of judging teacher quality by the rise or fall of test scores, and naturally she wondered what Secretary Arne Duncan thought about them.

There was the report of the American Statistical Association, which said: l

“VAMs should be viewed within the context of quality improvement, which distinguishes aspects of quality that can be attributed to the system from those that can be attributed to individual teachers, teacher preparation programs, or schools. Most VAM studies find that teachers account for about 1% to 14% of the variability in test scores, and that the majority of opportunities for quality improvement are found in the system-level conditions. Ranking teachers by their VAM scores can have unintended consequences that reduce quality.”

Days ago, a new Gates-funded study found no correlation between “quality teaching and the appraisals teachers received.”

Another study by a team led by Marianne P. Bitler, an economist at the University of California, said that VAM ratings had about the same relationship to reading and math scores as to changes in a student’s height.

VAM is the centerpiece of Race to the Top..

Strauss called the u.S. Department of Education to ask whether Secretary Duncan was aware of the research and whether it had changed his views. The answers: yes, he was aware of the research; no, it had not changed his views.