Archives for category: Teacher Evaluations

Since the announcement of Race to the Top, schools have become even more obsessed with test scores than they were under No Child Left Behind (which still exists in law). To be eligible to get a portion of Race to the Top’s $4.35 billion, states had to agree to evaluate their teachers to a significant degree by the test scores of their students. Later, when states needed waivers from NCLB’s mandate that every child must be proficient by 2014 or their schools would be labeled “failing,” Secretary Duncan made the waivers conditional on states’ agreement to evaluate teachers by the test scores of their students. Unfortunately at the time the U.S. Department of Education insisted on what was known as “value-added-modeling” (VAM), testing experts did not agree that it was fair or reliable as a measure of teacher effectiveness. On October 5, 2009, the Board on Testing and Assessment of the National Research Council wrote a letter to Secretary Duncan warning him of the pitfalls and limitations of VAM. Secretary Duncan ignored their advice and their caution. So now the entire nation is ensnared in an unscientific, unproven method to evaluate teachers.

 

 

John Ogozolek, who teaches in upstate New York, developed an analogy to try to make sense of VAM evaluations:

 

 

 

The analogy I was using this past week involves home repairs. Imagine your house needs a new roof or a new bathroom or even a new foundation. But then some big multi-national construction corporation shows up with a scheme of its own -not only to fix what your house might need but to renovate the entire place, top to bottom. Plus, they’re here to “fix” every other home in town, too -with the EXACT SAME “PLAN”.

 

So, you’re trying to have a rational discussion with the foreman of this alleged construction crew then you hear a chainsaw kick on down the hallway. You run down there just as some guy starts ripping apart your kid’s bedroom furniture, splinters flying. You can barely hear what the smiling worker is saying over the din but you get the idea when he hands you your own chainsaw and motions for YOU to join in, too. What the hell is the plan, you scream. “We…don’t….have….one”, he yells back, “We’re building the plane as we’re flying it”

 

WHAT….?

 

They are destroying your entire house. But, of course, it doesn’t really matter to them. If they burn down the town from one end to another, they’d only be happier. They don’t want you, they don’t want your house.

 

Their plan to renovate the world reminds me of the wacky “modern” architecture that leveled entire neighborhoods in cities 50 years ago and left us with mammoth housing projects….the urban “renewal” that ended up being worse than the tragic problems it sought to cure. It was all so much about science and “progress” back then, too, you know. I reread Tom Wolfe’s critique of modern architecture (“From Bauhaus to Our House”) a few years ago, when the Race to the Top crap first really hit the fan. How could such madness grip an entire culture and come to shape our world? I just had to shake my head…..yup, here we go again.

 

The statistical argument about VAM is great….it’s powerful……I’ve used it myself since reading about it on this blog. We should shout it from the rooftops. But I don’t think some of the people who are attacking public schools and destroying what goes on in our classrooms really care.

 

It’s like a good carpenter thinking about how to use a specific tool…..meanwhile a giant, smoke-belching excavator is sidling up on its greasy tracks, crushing whatever is in its way and swinging the good ole wrecking ball……

Peter Greene reports that the National Association of Secondary School Principals is reviewing and likely to endorse a statement rejecting VAM. The NASSP recognizes a growing body of research that shows the inaccuracy of VAM.

They cite the research, then offer recommendations:

“NASSP recommends that teacher eval include multiple measure, and that Peer Assistance and Review programs are the way to go. Teacher-constructed portfolios of student learning are also cool.

“VAMs should be used to fine tune programs and instructional methods as well as professional development on a building level, but they should not be “used to make key personnel decisions about individual teachers.” Principals should be trained in how to properly interpret and use VAMmy data.”

This is an important step forward, toward professional responsibilty and common sense.

Colorado has one of the very worst, most punitive educator evaluation laws in the nation, called SB 191. It was written by ex-TFA member State Senator Michael Johnston.

Please sign this petition to repeal 191.

In typical corporate reform fashion, the bill has a deceptive title,”like “Great Schools, Great Teachers,” but the mechanism of “greatness” is to tie 50% of teachers’ evaluations to student test scores. In 2010, when the bill was passed, value-added-assessment was new and promising. The Gates Foundation and the U.S. Department of Education promoted it. To be eligible for Race to the Top funding or for a waiver from the impossible mandates of No Child Left Behind, starred were required to evaluate teachers by their students’ scores.

Now we know that VAM doesn’t work. It is inaccurate , unreliable, and demoralizing. It says more about who is in the class than teacher quality.

It is time to get rid of VAM.

This is the executive summary of the statement of the American Statistical Association on the use of value-added assessment to evaluate teachers. Please share it with other teachers, with principals, and school board members. Please share it with your legislators and other elected officials. Send it to your local news outlets. The words are clear: Teachers account for between 1 and 14% of the variation in test scores. And this is very important to remember: “Ranking teachers by their VAM scores can have unintended consequences that reduce quality.”

 

 

ASA Statement on Using Value-Added Models for Educational Assessment

 

April 8, 2014

 

Executive Summary

 

Many states and school districts have adopted Value-Added Models (VAMs) as part of educational accountability systems. The goal of these models, which are also referred to as Value-Added Assessment (VAA) Models, is to estimate effects of individual teachers or schools on student achievement while accounting for differences in student background. VAMs are increasingly promoted or mandated as a component in high-stakes decisions such as determining compensation, evaluating and ranking teachers, hiring or dismissing teachers, awarding tenure, and closing schools.

 

The American Statistical Association (ASA) makes the following recommendations regarding the use of VAMs:

  • The ASA endorses wise use of data, statistical models, and designed experiments for improving the quality of education.
  • VAMs are complex statistical models, and high-level statistical expertise is needed to develop the models and interpret their results.
  • Estimates from VAMs should always be accompanied by measures of precision and a discussion of the assumptions and possible limitations of the model. These limitations are particularly relevant if VAMs are used for high-stakes purposes.

 

o VAMs are generally based on standardized test scores, and do not directly measure potential teacher contributions toward other student outcomes.

 

o VAMs typically measure correlation, not causation: Effects – positive or negative – attributed to a teacher may actually be caused by other factors that are not captured in the model.

 

o Under some conditions, VAM scores and rankings can change substantially when a different model or test is used, and a thorough analysis should be undertaken to evaluate the sensitivity of estimates to different models.

 

• VAMs should be viewed within the context of quality improvement, which distinguishes aspects of quality that can be attributed to the system from those that can be attributed to individual teachers, teacher preparation programs, or schools. Most VAM studies find that teachers account for about 1% to 14% of the variability in test scores, and that the majority of opportunities for quality improvement are found in the system-level conditions. Ranking teachers by their VAM scores can have unintended consequences that reduce quality.

Peter Greene noted that Minneapolis followed the terrible examples of Los Angeles in 2010 and New York City in 2012 and published teachers’ value-added ratings in the newspapers for all to see. Even Bill Gates objected to this practice and said in a New York Times article that it would harm the relationship between supervisors and teachers to publish job ratings in the paper. Gates said that publishing VAM scores was an act of “public shaming” and no good would come of it.

 

Greene writes:

 

As promised, this morning brought the publishing of teacher ratings, including VAM scores, with a map and a pearl-clutching interview with the district’s superintendent. The gap is shocking, alarming, inexplicable.

 

I’m speaking of course of the apparent gap between Superintendent Bernadeia Johnson’s brain and reality. How does somebody with this gigantic an inability to process data end up as a superintendent of a major school system?

 

Superintendent Johnson is shocked– shocked!!– to find that under this evaluation system, it turns out that all the worst teachers are working in all the poorest schools! Hmmm– the poorest schools have the worst results. What’s the only possible explanation? Teachers!! [Pause for the sound of me banging my head on the desk.]

 

“It’s alarming that it took this to understand where teachers are,” Superintendent Bernadeia Johnson said Friday. “We probably knew that, but now have the hard evidence. It made me think about how we need to change our staffing and retention.”

 

No, Superintendent Johnson. What’s alarming is that you don’t understand a damn thing.

 

Here’s what you have “discovered.” If you rip the roof off a classroom, the teachers that you send to teach in that classroom will get wet when it rains. You cannot “fix” that by changing the teacher.

 

But apparently that’s the solution being considered. “Okay,” says Superintendent Johnson. “Over here we have teachers who stay dry and their students stay dry, so we’ll put this dry teacher in the classroom without a roof and have a dry teacher for the wet rooms. That’ll fix it.”

 

And Superintendent Johnson appears willing to go further. “Maybe we just need to fire the wet teachers and replace them with new, dry ones,” she may be thinking. [Sound of me banging my head against the concrete slab of my basement floor.]

 

If you want a dry teacher in the room, build a damn roof on it.

 

Look. Look look look look look. We already know that poverty absolutely correlates with test results. Show me your tests results and I will show you where your low-income students are. Poverty and lack of resources and underfunding put these students in a classroom without a roof, and anybody you put in there with them will be a wet teacher.

 

Build a damn roof.

 

Minneapolis public school officials say they are already taking immediate action to balance schools’ needs with teachers’ abilities. The district has created programs to encourage effective instructors to teach at high-needs schools and mentor the newest teachers. District officials say they are providing immediate training for teachers who are deficient. And last year, the district fired more than 200 teachers, roughly 6 percent of its teaching staff.

 

Wrong. All wrong. In fact, worse than wrong, because you are now in the position of saying, “Hey, over here we have a room with no roof on it, and if you teach in there and get wet when it rains, we intend to punish you. Now– who wants to volunteer to teach in the roofless room?? Also, we’ll probably smear your good name in the local paper, too. Any takers?”

 

And to the students, sitting in that roofless room day after day, shivering and wet as poverty and lack of resources and insufficient materials and neglect by the central office rain down on them, this sends a terrible message. “We know you are sick and wet in your roofless room,” says the district. “So we are not sending a roof or even ponchos or an umbrella. We’re not going to spend a cent more on you. We’re just going to stand a different teacher up in front of you, to see if she gets wet when it rains.”

 

 

Last week, Kevin Huffman and John Ayers resigned. Huffman was state commissioner of education in Tennessee, and he employed every possible strategy to make testing a centerpiece of education policy. Ayers was director of the Cowen Institute at Tulane University in New Orleans, which was greatly embarrassed when it released–and then rescinded–a “research” report claiming amazing gains in the charter schools of New Orleans. Both were big boosters of using student test scores to judge the quality and effectiveness of teachers, a methodology referred to as VAM, or value-added-modeling.

 

Audrey Amrein-Beardsley, one of the nation’s expert researchers on teacher evaluation, looks at the two resignations as evidence that the VAM-mania is failing and claiming victims. There is as yet no evidence that VAM improves teaching,  improves student achievement, or correctly identifies the strengths and weaknesses of teachers. As its critics have said consistently, VAM results depend on many factors outside the control of the teacher and may vary for many different reasons. A teacher may get a high VAM rating one year, and a low VAM rating the next year. VAM ratings may change if a different test is used. Yet those who stubbornly believe that everything that matters can be measured with precision can’t let go of their data-driven mindset.

 

The lesson: proceed with caution with a methodology that has no record of success and that inevitably places far too much importance on standardized tests.

A few days ago, I posted about a plan by the Massachusetts Department of Elementary and Secondary Education (DESE) to take away teachers’ licenses if they received poor evaluations. Not just to take away their tenure or their job, but their license to teach. I relied on a terrific post by Peter Greene saying that Massachusetts had come up with an ingenious way to chase teachers out of the state. Given how flimsy and flawed the new test-based evaluations are, this was a horrendous plan that lacked logic, common sense, or basic decency. Given the fact that Massachusetts is by far the highest performing state on NAEP, these draconian measures were incomprehensible.

 

The Massachusetts Teachers Association rallied their members against the DESE plan, and the state DESE backed down. This is the MTA’s description of what happened, how they mobilized, and why good sense prevailed.

 

Here is the communique from the MTA leadership:

 

MTA MEMBERS SHOW UNION POWER; DESE RESCINDS PROPOSALS LINKING LICENSURE TO EDUCATOR EVALUATION

 

 

MTA President Barbara Madeloni and Vice President Janet Anderson sent the following message to MTA members on Friday, November 14:

 

We did it! In recent days, thousands of you have contacted state education officials to express your opposition to linking your license to your evaluation. MTA members sent e-mails, spoke out at DESE’s “town halls,” organized building meetings and made plans to attend upcoming DESE meetings in Malden and Bridgewater.

 

Today, the commissioner of education released a letter that says: “… we are rescinding the draft options that link licensure to educator evaluation.”

 

Our message — Union Strong — is making a difference.

 

While the immediate threat is lifted, there is much more to be done to make sure state officials hear what educators think we and our students need.

 

Here’s the background on the licensure story.

 

Twenty-five days ago, MTA received notice of licensure changes proposed by DESE that would connect performance evaluation to license renewal and advancement. These proposals and the façade of voice given within the DESE “town halls” exposed the deep disconnect between educators and the department. Union members spoke out resoundingly. Several members of the Board of Elementary and Secondary Education joined us in telling the commissioner they opposed this licensure plan.

 

The decision announced today is a good start, but other aspects of proposed licensure changes are still unsettled, and the disconnect between educators and DESE remains.

 

The commissioner has invited us to “continue the conversation.” Let’s do just that by showing up in Malden on Nov. 19 and Bridgewater on Nov. 20 to tell our stories, speak our truth, and reclaim public education.

 

Here are the details of the meetings next week:

 

DESE-sponsored Town Halls on Licensure

 

Wednesday, November 19
4:30-7 p.m. (arrive at 4:15 p.m.)
Malden High School
77 Salem Street
Malden

 

AND

 

Thursday, November 20
4:30-7 p.m. (arrive at 4:15 p.m.)
Bridgewater State University
Crimson Hall – Dunn Conference Room
200 East Campus Drive
Bridgewater

Even as we move forward with our plans to make our voices heard, this is a moment to celebrate our strength and acknowledge the hard work of our members on this crucial issue. So thank you, and let’s keep up the fight!

In solidarity,

Barbara and Janet

 

 

Katie Lapham teaches ESL classes in Brooklyn. What is really rotten in the schools, she writes, are the terrible tests that her first-graders must take. Their purpose is solely to evaluate the teachers. The tests were largely developmentally inappropriate. No teacher, she writes, would create such absurd tests.

She writes:

“Last month, it took me two and a half days to administer the 2014-2015 Grade 1 Math Inventory Baseline Performance Tasks to my students because the assessment had to be administered as individual interviews (NYCDOE words, not mine). The math inventory included 12 tasks, many of which were developmentally inappropriate. For example, in demonstrating their understanding of place value, first graders were asked to compare two 3-digit numbers using and =. Students were also asked to solve addition and subtraction word problems within 100.

“While I do not believe my students were emotionally scarred by this experience, they did lose two and a half days of instructional time and were tested on skills that they had not yet learned. It is no secret that NYC teachers and administrators view these MOSL tasks as a joke. Remember, they are for teacher rating purposes ONLY. “You want them to score low in the fall so that they’ll show growth in the spring,” is a common utterance in elementary school hallways. Also, there will be even more teaching-to-the-test as educators will want to ensure that their students are proficient in these skills before the administration of the spring assessment. Some of the first grade skills might be valid, but others are, arguably, not grade-level appropriate.

“The Grade 1 ELA (English-language Arts) Informational Reading and Writing Baseline Performance Task took less time to administer (four periods only) but was equally senseless, and the texts we were given had us shaking our heads because they resembled third grade reading material. In theory, not necessarily practice, students were required to engage in a non-fiction read aloud and then independently read an informational text on the same topic. Afterwards, they had to sort through a barrage of text-based facts in order to select information that correctly answered the questions. On day one, the students had to complete a graphic organizer and on day two they were asked to write a paragraph on the topic. Drawing pictures to convey their understanding of the topic was also included in the assessment.”

Lapham was surprised to learn that there is an alternative assessment that progressive schools use. She wonders why her school, in a poor neighborhood, was never informed about the option.

Superintendents in the Lower Hudson Valley area spoke critically of the state evaluation system for teachers and principals, called the Annual Professional Performance Review, in a meeting with the editorial board of the Journal-News. .

The evaluation system does not accurately identify teachers as effective or ineffective, and the State Education Department has been unwilling to listen to professionals in the field. The implementation has been as flawed as the implementation of Common Core. Both are tied to testing, and both derive from Race to the Top. The state received $700 million in Race to the Top funding but is likely to spend multiples of that amount to carry out its mandates. Since no part of Race to the Top was based on research, it is unlikely to produce good results. What it has produced is disruption, demoralization, outrage, and a vibrant anti-testing and anti-Common Core movement, led by parents.

Greta Callahan’s article about teaching kindergarten in Minneapolis went viral. She wrote her article in response to one that appeared in the same paper asserting the “worst teachers are in the poorest schools.” She teaches in one of the poorest schools, and she tells her story.

 

To those who parrot the false claim that low test scores are caused by “bad teachers,” she offers a counter-narrative. She explains the burdens suffered by her students and the stress of being evaluated by a rubric that makes no sense.

 

Let’s start with what it means to be a “good teacher.” As the article says: “The district uses three different tools to evaluate teachers: classroom observations, a student survey and student achievement data.” Let’s put that into the perspective of a Bethune kindergarten teacher.

 

• Classroom observations: We have four per year. The teacher receives points based on standardized criteria; the feedback is generally helpful. But these observations also involve the observer walking up to students and asking what they are doing. Even my 5-year-olds, who may have just started school, get asked this question. The student is supposed to regurgitate the “I can” statement that correlates to “Focused Instruction.” The usual response, though, is something along the lines of “math” or “Jaden took my crayon!”

 

If you were in my room, observing an observation, you would laugh. I promise.

 

• Student surveys: I administer a student survey once a year. My 5-year-olds have to circle their responses (even though they can’t read) to questions about their teacher and school. Have you been around a 5-year-old? They are adorable, spacey, loud and unfocused — and under no circumstances does this student survey make sense for them or to them.

 

• Student achievement data: Two to three times a year, our students are pulled out of our classrooms and tested by a stranger from the district. When she asks our kids to go into a separate room with her and gives them a test, most of them shut down. It’s intimidating to them. Some are asked to take this test in the middle of breakfast; others are tested right after recess. The inconsistency of when our children are tested creates a test that isn’t being measured consistently or accurately, in my opinion.

 

These are the “achievement data” that are referenced in the article. The scores are often low and rarely reflect the students’ actual achievements. My fellow teachers and I have plenty of conflicting and affirming evidence to support our students’ actual achievements, growth and knowledge. But this evidence is not considered when determining the effectiveness of a teacher.

 

Recruiting and retaining teachers at a high-poverty school present unusual challenges:

 

The retention rate of teachers at my school and others like it will not go up unless we have more incentive to stay — and more assistance to attempt to give our students an even chance.

 

At Bethune, many of our students are what most Americans would define as starving. At least a third are HHM (homeless/highly mobile), see violence in their homes or neighborhoods regularly and come to school with baggage many of us couldn’t imagine, let alone at age 5. Yet they are expected to meet the standards of kindergartners at upwardly mobile neighborhood schools like Burroughs and Hale. As far as the tests are concerned, a teacher is a teacher and a student is a student.

 

There are plenty of reasons why a teacher might not want to teach in a school like Bethune. Say, physical safety. Within the last two weeks, I have been slapped so hard in the face by a student that I had to seek emergency care; have been threatened by a student who said he was going to go home, come back and hurt me, because I wrote him up for hurting one of my kindergartners, and have broken up numerous fights. My fellow teachers and I have had parents threaten our safety more times than I can count — threats delivered on school property, in front of students. And, lest anyone be misinformed, there is no combat pay for working at a school like mine.

 

My children are happy to come to school and they are eager to learn. But sometimes they just lose it. A student will throw a chair across the room, or scissors at other students, or kick and punch me. It takes time, love and energy to find out why they are doing this. Many are imitating behaviors they see at home. Sometimes they have bottled-up feelings about something they have experienced and don’t know how to handle their anger. So, I teach them. I love them. I’m consistently there for them. I report their situation to Hennepin County all too often.

Many of our children do not have someone who will look over their work with them at night or take them to an activity. Our parents are generally very young and trying their best. It takes a village, but our village is poverty-stricken in every imaginable way.

 

Please read Greta Callahan’s article. She says succinctly what most teachers experience and know: Teaching is hard work. Low scores are caused not by “bad teachers,” but by terrible life circumstances that harm children and families. Of course, teachers should be evaluated, but not in the idiotic way she describes. Teachers who flounder need help and peer assistance. If they can’t teach, after sustained efforts to help them, they should leave teaching. But the narrative of “bad teachers cause low scores” is wrong. It ignores the effects of the single biggest blight on our society: growing poverty and inequality.