Archives for category: Teacher Evaluation

Jordan Weissman, a business correspondent for Slate, read the Vergara decision and noted that the judge’s conclusion hinged on a strange allegation. The judge quoted David Berliner as saying that 1-3% of the teachers in the state were “grossly ineffective.” The judge then calculated that this translated into thousands of teachers, between 2,750 and 8,750, who are “grossly ineffective.”

Weissman called Professor Berliner and asked where the number 1-3% came from. Dr. Berliner said it was a “guesstimate,”

He told Weissman, “It’s not based on any specific data, or any rigorous research about California schools in particular. “I pulled that out of the air,” says Berliner, an emeritus professor of education at Arizona State University. “There’s no data on that. That’s just a ballpark estimate, based on my visiting lots and lots of classrooms.” He also never used the words “grossly ineffective.” And he does not support the judge’s belief that teacher quality can be judged by student test scores.

Dr. Berliner mailed Weissman a copy of the transcript to show that he did not use the term “grossly ineffective.”

Weissman then called Stuart Biegel, a law professor and education expert at UCLA, to ask him “whether he thought that the odd origins of the 1–3 percent figure might undermine Treu’s decision on appeal. Biegel, who represented the winning plaintiffs in one of the key cases Treu cited, said it might. But he thought that the decision’s “poor legal reasoning” and “shaky policy analysis” would be bigger problems. “If 97 to 99 percent of California teachers are effective, you don’t take away basic, hard-won rights from everybody. You focus on strengthening the process for addressing the teachers who are not effective, through strong professional development programs, and, if necessary, a procedure that makes it easier to let go of ineffective teachers,” he wrote to me in an email.”

Bob Shepherd writes on the absurd demands now placed on teachers and principals by politicians, who expect to see higher test scores every year. Step back and you realize that the politicians, the policy wonks, the economists, and the ideologues are ruining education, not improving it. They are doing their best to demoralize professional educators. What are they thinking? Are they thinking? Or is it just their love of disruption, let loose on children, families, communities, and educators?

Bon Shepherd writes:

OK, you are sitting in your year-end evaluation session, and you’ve heard from every other teacher in your school that his or her scores were a full level lower this year than last, and so you know that the central office has leaned on the principal to give fewer exemplary ratings even though your school actually doesn’t have a problem with its test scores and people are doing what they did last year but a bit better, of course, because one grows each year as a teacher–one refines what one did before, and one never stops learning.

But you know that this ritual doesn’t have anything, really, to do with improvement. It has to do with everyone, all along the line, covering his or her tushy and playing the game and doing exactly what he or she is told. And, at any rate, everyone knows that the tests are not particularly valid and that’s not really the issue at your school because, the test scores are pretty good because this is a suburban school with affluent parents, and the kids always, year after year, do quite well.

So whether the kids are learning isn’t really the issue. The issue is that by some sort of magic formula, each cohort of kids is supposed to perform better than the last–significantly better–on the tests, though they come into your classes in exactly the same shape they’ve always come into them in because, you know, they are kids and they are just learning and teaching ISN’T magic. It’s a lot of hard work. It’s magical, sometimes, of course, but its’ not magic. There’s no magic formula.

So, the stuff you’ve been told to do in your “trainings” (“Bark. Roll over. Sit. Good Boy”) is pretty transparently teaching-to-the-test because that’s the only way the insane demand that each cohort will be magically superior to the last as measured by these tests can be met, but you feel in your heart of hearts that doing that would be JUST WRONG–it would short-change your students to start teaching InstaWriting-for-the-Test, Grade 5, instead of, say, teaching writing. And despite all the demeaning crap you are subjected to, you still give a damn.

And you sit there and you actually feel sorry for this principal because she, too, is squirming like a fly in treacle in the muck that is Education Deform, and she knows she has fantastic teachers who knock it out of the park year after year, but her life has become a living hell of accountability reports and data chats to the point that she doesn’t have time for anything else anymore (she has said this many times), and now she has to sit there and tell her amazing veteran teachers who have worked so hard all these years and who care so much and give so much and are so learned and caring that they are just satisfactory, and she feels like hell doing this and is wondering when she can retire.

And the fact that you BOTH know this hangs there in the room–the big, ugly, unspoken thing. And the politicians and the plutocrats and the policy wonks at the Thomas B. Fordham Institute and the Secretary of the Department for the Standardization of US Education, formerly the USDE, and the Vichy education guru collaborators with these people barrel ahead, like so many drunks in a car plowing through a crowd of pedestrians.

Audrey Amrein-Beardsley, one of our nation’s pre-eminent experts on value-added assessment, here reviews a TED-X talk by Tennessee Commissioner of Education Kevin Huffman, boasting of the tremendous growth in test scores as a result of his policies. Beardsley points out the curious fact that Tennessee started using VAM in the 1990s with little to show for it. But, there were those Tennessee NAEP scores, proof positive, according to both Huffman and Se rotary of Education Arne Duncan that Race to the Top–or Huffman’s personal presence–was creating strong results. Nd in the end, results (test scores) are what matter most, right?

But what about those NAEP results that Huffman and Duncan tout?

Beardsley writes:

“While [William] Sanders (the TVAAS developer who first convinced the state legislature to adopt his model for high-stakes accountability purposes in the 1990s) and others (including U.S. Secretary of Education Arne Duncan) also claimed that Tennessee’s use of accountability instruments caused Tennessee’s NAEP gains (besides the fact that the purported gains were over two decades delayed), others have since spoiled the celebration because 1) the results also demonstrated an expanding achievement gap in Tennessee; 2) the state’s lowest socioeconomic students continue to perform poorly, despite Huffman’s claims; 3) Tennessee didn’t make gains significantly different than many other states; and 4) other states with similar accountability instruments and policies (e.g., Colorado, Louisiana) did not make similar gains, while states without such instruments and policies (e.g., Kentucky, Iowa, Washington) did. I should add that Kentucky’s achievement gap is also narrowing and their lowest socioeconomic students have made significant gains. This is important to note as Huffman repeatedly compares his state to theirs.”

Read the post. It is a very good demonstration of how data get used and misused for political purposes.

Four years ago, I was in Colorado to discuss education policy. This was in the heady early days of Race to the Top (which Colorado did not win, despite its whole-hearted embrace of everything Arne Duncan wanted). On one occasion, I was scheduled to debate State Senator Michael Johnston, the darling of the “reform” crowd. Johnston had written a bill that was coming to a vote that very day. His bill made student test scores count for 50% of every educators’ evaluation. An effective evaluation, his bill decreed, required growth in student scores. Johnston called his bill something like “Great Schools, Great Educators.” Or something like that. Every bill these days must contain at least one impossible promise in its title.

As I said, we were supposed to debate in front of a packed room of civic leaders, maybe 80 or so people. I waited and waited. No Johnston. Finally, I got up and spoke my concerns in his absence. No sooner did I finish than the doors at the back of the room opened and out popped young Senator Johnston. I say young because he appeared to be about 25, though I think he was actually 32. He was then considered the leading voice of education reform in the Legislature, despite members who were retired and experienced educators. Senator Johnston had served two years in Teach for America, then was principal of a school for two years, then ran for state senate. And now he was rewriting the state’s education laws! Truly a whiz kid!

Since he did not hear me, he did not have to respond to anything I said. Instead, he spoke in glowing terms of his legislation. He had an almost mystical faith in the amazing results that would automatically materialize as soon as teachers and principals were evaluated by the academic growth of their students. He seemed to believe that the only source of low scores was the absence of incentives and sanctions for those unmotivated, possibly lazy educators. Everyone, it seemed, wanted to believe that he knew what he was talking about.

Now, we know it takes time to phase in new policies and practices. As Bill Gates famously said, “It will take a decade to know whether this stuff works.” What he meant by “this stuff,” I guess, is the idea that privatization and measuring teacher quality by student scores will make students better educated. My own view is that we should stop looking for the “secret sauce” because it is a chimera. Instead, we should do what we know works, which is reduced class sizes, early childhood education, family education, experienced teachers, healthy children, a full and rich curriculum, and the wraparound services that children need. But all that is complicated, not simple; our data-driven reformers like simple solutions, the bumper sticker ideas.

But surely we should see some positive movement in Colorado, don’t you think? And it should be cumulative, stronger every year as the “reforms” take hold.

The latest state scores from Colorado–which has been dominated by data-driven reformers for a decade– are unimpressive. Actually, the scores of third-graders, who have known nothing other than a testing culture, took a slight dip. In truth, they were flat.

Oh, well, maybe next year, we will see the miracle that Senator Johnston promised. Or the year after that.

Meanwhile Senator Johnston has been invited to be Alumni Commencement Speaker at Harvard Graduate School of Education, which has aroused some protest. This is allegedly a tribute to his great accomplishment in Colorado, where every year his promises grow more hollow. How many of the graduates at HGSE would want to work under Johnston’s law? Presumably, students at HGSE read research and know that VAM is Junk Science.

Barbara Madeloni, who led the fight against outsourcing teacher credentialing to Pearson, was elected president of the Massachusetts Teacher Association, will take charge of a union of 110.000 educators

“Until last August, Madeloni directed the Secondary Teacher Education Program at the University of Massachusetts.

“While UMass said her employment ended as part of a move to reduce the use of adjunct professors, Madeloni stated in interviews that the school was punishing her for opposing a project in which UMass tested a teacher assessment program for the for-profit company Pearson.

“Madeloni, 57, said in an interview Sunday she plans as MTA president to “amplify the voice of educators and be a leader at the national level.”

“She noted that her victory comes amid efforts in Los Angeles, Seattle and Chicago to shift the debate back to supporting high-quality public education and the people who provide it over the interests of for-profit companies in the field.

“It should be national news,” Madeloni said of her win in Massachusetts. “It’s a message to everybody that teachers will not be silent and compliant as this assault on public education continues — and undermines public education. This is foundational to democracy and we need to defend it.”

Jonathan Pelto here reports on a great new piece by civil rights lawyer Wendy Lecker.

 

He writes:

“In her latest MUST READ commentary piece, fellow public education advocate, Wendy Lecker, lays out the facts about Governor Malloy’s unfair, inappropriate and fatally flawed teacher evaluation system. Like the junk bonds that helped take down Wall Street, Connecticut’s teacher evaluation system is based on junk science and false assumptions.

 

The question is not whether the state should have a comprehensive teacher evaluation system, but whether the corporate education reform industry will continue to stand in the way of developing one.

 

Lecker says that Governor Dannell Malloy’s teacher evaluation system is fundamentally flawed.

 

Lecker writes that the solution to failed tests is not more tests.

 

From her article:

 

Fact: Connecticut’s teacher evaluation plan, because it relies on student standardized test scores, is fundamentally flawed. Student test scores cannot measure a teacher’s contribution to student learning. In fact, the president of the Educational Testing Service recently called evaluation systems based on student test scores “bad science.”

 

Rather than admit failure, the Malloy administration is trying futilely to “fix” the fatal flaw. Last week, PEAC, the panel charged with developing Connecticut’s teacher evaluation system, working under the direction of Commissioner Stefan Pryor, approved a change which calls for more standardized tests to be included in a teacher’s evaluation.

 

The commissioner’s “solution” is to add interim tests to a teacher’s rating. Determining what tests will be used, how they will be aligned to the standardized tests, and how all the test scores will be rolled into one “score” for teachers, will likely render this change completely unworkable.

 

She adds:

 

A recent comprehensive study by Northwestern Professor Kirabo Jackson found that children with teachers who help them develop non-cognitive skills have much better outcomes than those who have teachers who may help them raise test scores. Jackson found that every standard deviation increase in non-cognitive skills corresponds to a significant decrease in the drop-out risk and increased rates of high school graduation. By contrast, one standard deviation increase in standardized test scores has a very weak, often non-existent, relationship to these outcomes. Test scores also predict less than 2 percent of the variability in absences and suspensions, and under 10 percent of the variability in on-time grade progression, for example.

Increases in non-cognitive abilities are also strongly correlated with other adult outcomes, such as a lower likelihood of arrest, a higher rate of employment and higher earnings. Increased test scores are not.

In short, focusing on non-cognitive abilities, those not measured by test scores, are more important in predicting success in high school and beyond.

 

Why are the corporate reformers so wedded to standardized tests that they themselves probably could not pass? They love data. They want Big Data. They believe that every problem can be solved by measurement and manipulation of Big Data. They also believe that they can create the appearance of “failing public schools” by generating data showing how many kids are not meeting an artificial benchmark. This enables them to argue for more charter schools that are free to exclude the children who did not meet the artificial bench mark. Big Data is now part of the tool kit of privatization. It is not about helping kids or improving education, but finding a rationale for turning public dollars over to private managers. If we really wanted to help kids and improve education, we would take the billions now going into testing and use it to reduce class sizes, to increase the arts, and to provide every child the medical care they need.

 

Washington State thoughtfully rejected Arne Duncan’s threat to cancel its waiver from the absurd demands of No Child Left Behind. The decision to say no to federal demands and intimidation was bipartisan.

The Legislature refused to bend to Duncan’s insistence that the state adopt test-based evaluation, which has consistently failed across the nation and has been declared inaccurate by the nation’s leading scholarly organizations.

The Washington State legislature understands federalism. Secretary Duncan does not. He thinks he is charge of the nation’s schools–every one f them. As someone who spent eight years running the Chicago public school system, one of the nation’s lowest-performing, he should have earned humility. Unfortunately, he enjoys a sense of certainty that is astonishing, almost as astonishing as his indifference to research and evidence.

The sense of the Washington State legislature was succinctly expressed by Chris Rekydal, a Democrat.

Unlike Duncan, Rekydal understands that the Tenth Amendment to the Constitution leaves education policy to states and localities.

He said in a statement:

“As a legislator who voted for our state’s robust home-grown teacher-principal evaluation system and one of the authors of our state’s new rigorous 24-credit graduation framework, I am disappointed in the federal government’s decision to repeal our waiver.

“This is a tremendous moment in our nation’s history where a state that strongly supported the President in 2008 and again in 2012 soundly rejected the federal government’s demands to structure our teacher-principal evaluation system to the specific criteria established by the U.S. Dept. of Education.

“My message to President Obama and Secretary Duncan is that Washington State is committed to education reform that is collaborative, bipartisan, and focused on student success and teacher growth. Our legislative decision to reject the federal government’s demands was done with substantial deliberation and a deep respect for state and local control.

“The bipartisan rejection of this federal government demand during the 2014 legislative session is a strong and unifying message that our state fully embraces our constitutional 10th Amendment guarantee to develop, fund, and administer our state’s education system as the citizens of the state of Washington and their elected representatives determine, not as federal officials deem it appropriate.

“Washington State has one of the leading K-12 systems in the United States. With 89% of our adult population having earned a high school diploma or greater, we are a national leader in student success, employment growth, and earnings.

“I strongly encourage federal officials to use this moment in history to model Washington State’s success instead of using us as an example of federal government power and leverage. I challenge the federal government to turn a corner on education reform, fix the deeply-flawed and failed No Child Left Behind Act, and get back to empowering the states instead of coercing them.

“No Child Left Behind is a failed policy of the Bush administration that focuses on student failure and school punishment. This is no way to run a public education system. Enacting bad policy at the state level as a result of bad policy at the federal level will not help schools – and certainly won’t help students – be successful.”

In a stunning reversal,the Tennessee Legislature overwhelmingly repealed a law to evaluate teachers by test scores, and the law was swiftly signed by Governor Haslam. On a day when Arne Duncan withdrew Washington State’s failure to enact test-based teacher valuation system, this is a remarkable turn of events.

Joey Garrison of The Tennessean reports:

“Gov. Bill Haslam has signed into law a bill that will prevent student growth on tests from being used to revoke or not renew a teacher’s license — undoing a controversial education policy his administration had advanced just last summer.

“The governor’s signature, which came Tuesday, follows the Tennessee General Assembly’s overwhelming approval this month of House Bill 1375 / Senate Bill 2240, sponsored by Republicans Rep. John Forgety and Sen. Jim Tracy, which cleared the House by a unanimous 88-0 vote and the Senate by a 26-6 vote.

“That marked a major repudiation of a policy the Tennessee Board of Education in August adopted — at Education Commissioner Kevin Huffman’s recommendation — that would have linked license renewal and advancement to a teacher’s composite evaluation score as well as data collected from the Tennessee Value-Added Assessment System, which measures the learning gains of students.

“The bill to reject the policy had been pushed chiefly by the Tennessee Education Association, the state’s largest teachers’ organization, which engineered a petition drive to encourage Haslam to sign the legislation despite it passing with large bipartisan support.

“Huge, huge win for teachers,” the TEA wrote on its Twitter page, thanking both bill sponsors as well as Haslam for “treating teachers as professionals.”

“Eyeing a 2015 implementation, the state board in January had agreed to back down from using student learning gains as the sole and overriding reason to revoke a license. Composite evaluation scores, in which 35 percent is influenced by value-added data, were to centerpiece.”

********************
Two interesting points here: one, Duncan has been hailing Tennessee as a demonstration of the “success” of Race to the Top, in which test-based evaluation of teachers is key. What happens now?

Second, state Commissioner Kevin Huffman is so unpopular that anything he supports is likely to be rejected. His enemies hope he doesn’t leave Tennessee because whatever he recommends generates opposition, even among his allies.

Laura H. Chapman left the following comment. The word “desperate” to describe this quest for a scientific, data-based means of judging teachers is mine. Something about it smacks of anti-intellectualism, the kind of busywork exercise that an engineer would design, especially if he had never taught K-12. This is the sort of made-up activity that steals time from teaching and ultimately consumes a lot of time with minimal rewards.

Chapman writes:

Please give at least equal attention to the 70% of teachers who have job assignments without VAMs (no state-wide tests). For this majority, USDE promotes Student Learning Objectives (SLOs) or Student Growth Objectives (SGOs), a version of 1950s management-by-objectives on steroids.

Teachers who have job-alike assignments fill in a template to describe an extended unit or course they will teach. A trained evaluator rates the SLO/SGO (e.g. “high quality” to “unacceptable” or “incomplete”).

The template requires the teacher to meet about 25 criteria, including a prediction of the pre-test to post-test gains in test scores of their students on an approved district-wide test. Districts may specify a minimum threshold for these gains.

Teachers use the same template to enter the pre-and post-test scores. An algorithm determines if the gain meets the district threshold for expectations, then stack ranks teachers as average, above or below average, or exceeding expectations.

1. The Denver SLO/SGO template is used in many states. This example is for art teachers—-Denver Public Schools. (2013). Welcome to student growth objectives: New rubrics with ratings. http://sgoinfo.dpsk12.org/
2. One of the first attempts to justify the use of SLOs/SGOs for RttT—-Southwest Comprehensive Center at WestEd (n.d.). Measuring student growth in non-tested grades and subjects: A primer. Phoenix, AZ: Author. http://nassauboces.org/cms/lib5/NY18000988/Centricity/Domain/156/NTS__PRIMER_FINAL.pdf

3. This USDE review shows that SLOs/SGOs have no solid research to support their use—-Gill, B., Bruch, J., & Booker, K. (2013). Using alternative student growth measures for evaluating teacher performance: What the literature says. (REL 2013–002). Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Mid-Atlantic. http://ies.ed.gov/ncee/edlabs.

4. The USDE marketing program on behalf of SLOs/SGOs—-Reform Support Network. (2012, December). A quality control toolkit for student learning objectives. http://www2.ed.gov/about/inits/ed/implementation-support-unit/tech-assist/slo-toolkit.pd

5. The USDE marketing campaign for RttT teacher evaluation and need for district “communication SWAT teams” (p. 9) —- Reform Support Network. (2012, December). Engaging educators, Toward a New grammar and framework for educator engagement. Author. http://www2.ed.gov/about/inits/ed/implementation-support-unit/tech-assist/engaging-educators.pdf

6. Current uses of SLOs/SGOs by state—-Lacireno-Paquet, N., Morgan, C., & Mello, D. (2014). How states use student learning objectives in teacher evaluation systems: a review of state websites. Washington, DC: US Department of Education, Institute of Education Sciences. http://ies.ed.gov/ncee/edlabs/regions/northeast/pdf/REL_2014013.pdf

7. Flaws in the concepts of “grade-level expectation” and “a year’s worth of growth” —-Ligon, G. D. (2009). The optimal reference guide: Performing on grade level and making a year’s growth: Muddled definitions and expectations, growth model series, Part III. Austin, TX: ESP Solutions http://www.espsolutionsgroup.com/espweb/assets/files/ESP_Performing_on_Grade_Level_ORG.pdf

A few days ago, I published a post about a paper by Kirabo Jackson, explaining that the non-cognitive effects of teachers are often more important than the test scores of their students.

 

As it happened, mathematician Robert Berkman read the paper and explains here why it is another nail in the coffin of value-added measures, which judge teacher quality by the rise or fall of student test scores.

 

Berkman writes:

 

In this post, I’m going to examine one of the studies that no doubt had a profound impact on the members of AMSTAT that led them to this radical (but self-evident) conclusion. In 2012, the researcher C. Kirabo Jackson at Northwestern University published a “working paper” for the National Bureau of Economic Research, a private, nonprofit, nonpartisan research organization dedicated to promoting a greater understanding of how the economy works (I’m quoting here from their website.) The paper, entitled “Non-Cognitive Ability, Test Scores, and Teacher Quality: Evidence from 9th Grade Teachers in North Carolina” questions the legitimacy of evaluating a teacher based on his/her students’ test scores. Actually, it is less about “questioning” and more about “decimating” and “annihilating” the practice of VAM.

 

He adds:

 

What should be noted is that Jackson is not an educational researcher, per se. Jackson was trained in economics at Harvard and Yale and is an Associate Professor of Human Development and Social Policy. His interest is in optimizing measurement systems, not taking positions on either side of the standardized testing debate. Although this paper should reek with indignation and anger, it makes it’s case using almost understated tone and is filled with careful phrasing like “more than half of teachers who would improve long run outcomes may not be identified using test scores alone,” and “one might worry that test-based accountability may induce teachers to divert effort away from improving students’ non-cognitive skills in order to improve test scores.”

But lets get to the meat of the matter, because this paper is 42 pages long and incorporates mind-boggling statistical techniques that account for every variable one might want to filter out to answer the question: are test scores enough to judge the effectiveness of a teacher? Jackson’s unequivocal conclusion: no, not even remotely.

 

The only puzzle is why Arne Duncan keeps shoving VAM down the throats of states and school districts.

 

 

PS: Berkman added his credentials in a comment:

“I’m a math teacher who has worked with pre-K through college aged students for 30 years. My degrees are in Urban Studies, and Elementary Math Education. I have also done extensive work in neuroscience and numeracy, as well as technology and education, not to mention cognitive science.”
 

 

 

Follow

Get every new post delivered to your Inbox.

Join 104,848 other followers