Audrey Amrein-Beardsley looks closely at the AERA statement on the use of value added modeling (VAM).
She reviews the complex statistical requirements for proper implementation. No state or school district today could meet these standards.
Audrey Amrein-Beardsley looks closely at the AERA statement on the use of value added modeling (VAM).
She reviews the complex statistical requirements for proper implementation. No state or school district today could meet these standards.
Bill Gates gave a major national speech yesterday, announcing that he was very pleased with his efforts to improve teaching in America, even though they had produced no results other than a national teacher shortage. He promised to stay the course.
Peter Greene here presents the gist of Bill’s speech to the nation.
“It’s been fifteen years since we started trying to beat public education into submission with giant stacks of money, and it turns out that it’s a hell of a lot harder than curing major diseases. Turns out teachers are not nearly as compliant as bacteria. Who knew?
“Actually, there’s a whole long list of things that came as a surprise to us. Teachers and politicians and parents all had ideas about what ought or ought not to be happening in schools, and damned if they would just not shut up about it. At first stuff was going great and we were getting everyone to do just what we wanted them to, but then it was like they finally noticed that a bunch of clueless amateurs were trying to run the whole system, and the freaked out.
“I have to tell you. Right now as I’m sitting here, it still doesn’t occur to me that all the pushback might be related to the fact that I have no educational expertise at all, and yet I want to rewrite the whole US school system to my own specs. Why should that be a problem? I still don’t understand why I shouldn’t be able to just redo the whole mess without having to deal with unions or professional employees or elected officials. Of course nobody elected me to do this! I don’t mind, really– happy to take over this entire sector of the government anyway, you’re welcome…..
“Look, I’m a simple man. I had some ideas about how the entire US education system should work, and like any other citizen, I used my giant pile of money to impose my will on everyone else. It’s okay, because I just want to help. We’re not done yet– I’m going to keep trying to fix the entire teaching profession, even if nobody in the country actually asked me to do it. And no, I don’t intend to talk to anybody actually in the profession. What do they know about teaching? Besides, when you know you’re right, you don’t have to listen to anybody else.”
Anthony notes that teacher evaluations have changed in most states and districts because of Race to the Top. He and others are conducting a study to see how teacher evaluation is working or not working.
Please read his post and respond to the survey, if you are so inclined.
Audrey Amrein-Beardsley testified on behalf of the plaintiffs (the teachers) in the court case against New Mexico’s teacher evaluation system. She is an expert on teacher evaluation and has had the benefit of having been a teacher. Her blog “Vamboozled” regularly criticizes the misuse of test-based evaluations programs (like VAM, value-added measurement) that use the rise or fall of student test scores as their measure of teacher effectiveness.
In this post, she gives an overview of day three of the trial. The main “expert witness” for the state, testifying in favor of VAM, was Tom Kane of Harvard. He previously directed the Gates Foundations MET (Measures of Teacher Effectiveness) study, which promoted the use of VAM.
It is noteworthy that neither Beardsley nor Kane was able to analyze New Mexico’s data because the state did not release them or make them available, even to its own “expert witness.”
Kane admitted that he
had not examined any of New Mexico’s actual data. This was surprising in the sense that he was actually retained by the state, and his lawyers could have much more likely, and literally, handed him the state’s dataset as their “expert witness,” likely regardless of the procedures and security measures (but perhaps not the timeline) I mentioned prior. Also surprising was that Kane had clearly not examined any of the exhibits submitted for this case, by both the plaintiffs and the defense, either. He was essentially in attendance, on behalf of the state, to “only speak to [teacher] evaluations in general.” As per an article this morning in The Albuquerque Journal, as an “expert witness” he “stressed that numerous studies show that teachers make a big impact on student success,” expressly contradicting the American Statistical Association (ASA), while referencing studies of primarily his econ-friends (e.g. Raj Chetty, Eric Hanushek, Doug Staiger) and those of his own (e.g, as per his Measures of Effective Teaching (MET) studies), although this latter (unambiguous) assertion was not highlighted in this article. For more information in general, though, see the articles this morning in both The Albuquerque Journal and The Santa Fe New Mexican.
Then the state called the superintendent of the Roswell Independent School District to testify in favor of the state’s evaluation model. He said that the new system was an improvement over the old one. He also testified that he would not use the ratings to fire teachers, because he already had a teacher shortage. He told the local newspaper that:
“I am down teachers. I don’t need teachers, number 1, quitting over this and, number 2, I am not going to be firing teachers over this.” His district of about 600 teachers currently has approximately 30 open teaching positions, “an unusually high number;” hence, “he would rather work with his current staff than bring on substitutes” in compliance. So while he testified on behalf of the state, he also testified he was not necessarily in favor of the consequences being attached to the state’s teacher evaluation output, even if as currently being positioned by the defense as “low-stakes.”
Hillsborough County in Florida was one of the major beneficiaries of the Gates Foundation’s fetish for teacher evaluation and bonus pay. Gates pledged “up to” $100 million, but is refusing to pay the last $20 million because there has been so little evidence of the link between bonuses and test scores. Duh. If the Gates Foundation read the research on incentive pay, it would have spent the money reducing class sizes for the neediest children.
The Gates program has cost a total of $271 million, including Gates’ $80 million.
The Hillsborough plan inspired state legislation:
“Enacted a year after Hillsborough launched its project, Senate Bill 736 in the Florida Legislature phased out teacher tenure and tied pay to supervisor evaluations and student test scores.”
The program never met its goal of firing 5% of teachers every year:
“The original proposal and a 2010 timeline called for the district to fire 5 percent of its teachers each year for poor performance. That would amount to more than 700 teachers. The thinking was they would be replaced by teachers who earned entry level wages, freeing up money to pay the bonuses for those at the top.
“But the mass firings never happened. While an undetermined number of teachers resign out of dissatisfaction or fear that they will be fired, only a handful of terminations happen because of bad evaluations.”
The Gates Foundation has another flop.
MaryEllen Elia, the superintendent of the Hillsborough school district when it received the Gates grant,, was fired by the school board, then hired this year as state superintendent in Néw York.
“Late in the process, the foundation rejected several of the district’s funding requests for Empowering Effective Teachers, which involves evaluating teachers using specially trained peers and bumping their pay with the idea that it would boost student performance.
“Each of the proposals were robustly outlined and presented,” a district report said.
“But Gates officials responded by pointing to language in the original agreement saying the foundation had promised “up to” $100 million, not necessarily the whole amount, according to the report.
“The district picked up the unpaid costs.
“Much of the disagreement amounted to a change in Gates’ philosophy, Brown said. “After a few years of research,” she said, “they believed there was not enough of a connection between performance bonuses and greater student achievement.”
Now for some laughs, enjoy Peter Greene’s take on Gates’ cancellation of $20 million. He reminds us that Hillsborough was a jewel in Gates’ crown in 2012.
“Well, that was 2012. A few other things have happened in the meantime. Back in 2010, Arne Duncan and Dennis Van Roekel stopped by to make a fuss, but that was about the last time that anybody wanted to throw an EET party.
“That fire 5% of the sucky teachers thing? It should have gotten rid of 700 (700!!!) teachers– you know, the expensive ones, because everyone knows that the bad teachers that need to be rooted out are, coincidentally, the older teachers who cost a bunch of money. But it never happened.
“And that $100 million grant that Kinser was so proud of? Funny thing. Gates officials would now like you to know that the grant actually said “up to” $100 million.
“I am kind of excited about that, because I know realize that I can tell, say, a used car dealer that I will pay “up to” seventy grand for a car and just pay five thousand bucks. I could promise to buy a new house with “up to” $10 million and just fork over a check for $10.75. I do regret not knowing this trick when my children were young and I could have bribed them to do chores with offers of “up to” $100 for mowing the lawn.”
Now for a deep analysis, read Mercedes Schneider’s analysis of the Hillsborough debacle. The Gates money was a Trojan horse. Not only did it fail to produce a new generation of super-teachers, it drained the district’s reserves.
The Gates money–$80 million, not the promised $100 million–was a cause of great celebration when it was announced. Hillsborough would be a “national model.” In the end, Superintendent Elia was fired in January 2015, the district lost millions, and Gates learned…what?
“Of course, Gates had some ideas about how this “teacher effectiveness” business should work. The report linked above has as its second sentence, “A teacher’s effectiveness has more impact on student learning than any other factor under the control of school systems, including class size, school size, and the quality of after-school programs.” When pro-corporate-reform organizations toss around such statements, they never seem to follow it with the fact that factors external to the classroom hold far more sway that does the teacher. (In analyzing the proportion of teacher influence captured via value-added modeling– VAM– the American Statistical Association notes that teacher influence accounts for between 1 and 14 percent of variance in student test scores. Thus, between 86 and 99 percent of a student’s test score is out of the teacher’s control.)
“Nevertheless, ignoring that the teacher controls so little of student outcomes in the form of market-driven-reform-loving test scores, in its efforts to try to purchase higher student test scores, the Gates Foundation offered ten school districts nationwide the multi-million-dollar-funded opportunity to prove that teachers could indeed be cajoled into producing better “student achievement” (i.e., ever-higher test scores) when such teachers were measured by their students’ test scores and offered more money for “raising” said scores.
“As a 2009 winner of an Empowering Effective Teachers grant, Hillsborough was thrilled (“We’ll be a national model!”). A December 21, 2015 archive of Hillsborough schools’ “Empowering Teachers” webpage includes a number of enthusiastic responses regarding the newly-acquired, $100 million Gates grant. Front and center in these celebratory public statements is then-Hillsborough superintendent, MaryEllen Elia (Then-Governor Charlie Crist: “I commend Superintendent MaryEllen Elia and the Hillsborough County School District for their enthusiasm and commitment to working with the Bill and Melinda Gates Foundation during the next seven years to improve student academic performance through rewarding high quality teachers both professionally and monetarily. The foundation’s generous grant award of $100 million will greatly enhance the work the district has already done in this area.”)
“However, part of the Hillsborough-Gates agreement involved Hillsborough’s ponying up money of its own– which ended up eating into the Hillsborough schools’ reserves and threatening its bond rating. As reported in the August 04, 2015, Tampa Bay Tribune, the Empowering Effective Teachers initiative is not the only financial stressor affecting the Hillsborough bond rating, but it is nevertheless noteworthy.”
How many more such defeats can the reformers take before they figure out that their ideas are failures?
The Hudson Valley Alliance for Public Education issued a statement pledging to increase the number if opt outs next spring, in response to the Board of Regents’ decision to endorse a punitive, test-based teacher evaluation system. In the new system, test scores will count for 50% of a teacher’s evaluation, despite the American Statistical Association’s warning that a teacher accounts for 1-14% of variation in test scores.
Supporters of VAM think that the teachers’ union is pulling the strings and persuading parents to express such views. They think parents are dupes and fools. They are wrong.
Parents understand that when test scores matter so much, teachers will spend more time on test prep and less time on untested subjects. This is educationally unsound, and it hurts their children.
The parents support opt out to protect their children.
Here is the statement of the Hudson Valley Alliance:
“Parents across the Hudson Valley are dismayed by yesterday’s vote by the Board of Regents to adopt teacher evaluation regulations that will double down on high stakes testing and the harmful effects of test-prep driven education. While we applaud the courage of those Regents who voted no, Hudson Valley parents are disappointed with Regent Finn’s failure to protect public school children in our area.
“Under Chancellor Tisch’s leadership, the Regents majority have failed to challenge flawed legislation that harms public school children” said Carol Newman Sharkey, Orange County public school parent. “It is clear that Chancellor Tisch must be removed from her position when her term is up this year.”
“The Regents failed to rise up against the Governor’s tyrannical demands and instead have allowed bad education policies to displace whole child and sound pedagogical practices. They have stood idly by while Cuomo makes a mockery out of public schools putting cronies, political ambition, and charter schools above children” said Tory Lowe, co-founder of Kingston Action for Education and Ulster County public school parent.
“This vote ensures that the opt out movement will continue to grow. Parents seeking to protect their children will not back down or be appeased by false promises of better tests. At the end of the day, you cannot measure teaching and learning with a test score. Until there is real change, parents will continue to reject a corrupt system that destroys authentic teaching and learning” said, Bianca Tanis, New Paltz public school parent.
Since the adoption of the Common Core-aligned assessments, the Regents have voted to limit the number of students entitled to extra support in the form of Academic Intervention Services while simultaneously labeling teachers and students as failures.
“Once again, NYSED seems to talk out of both sides of its mouth. The message that SED continues to spread is that almost 70% of the students in grades 3-8 aren’t “proficient”, but yet schools don’t have to provide AIS (i.e. – “flexibility”) if their level of failure isn’t low enough. Either our children who are scoring ‘1s and 2s’ on the state tests are struggling and they deserve to get the academic support to help them meet the standards, or the standards themselves are inappropriate. They cannot have it both ways” said Tim Farley, Columbia County public school parent.
Stacey Kahn, Ulster County public school parent said “We suggest that Chancellor Tisch and Commissioner stop insulting the intelligence of the public. We will refuse the tests until the Regents majority starts making decisions that put children before politics and corporate sponsors.”
“What took place at the Regents meeting only underscores what parents and educators have known for quite some time – Chancellor Tisch must go. It is critical that parents, educators, and concerned community members turn their eyes towards our state legislators who have the power to amend destructive education law and remove Chancellor Tisch and some of her colleagues as they seek reappointment in the new year. New York students deserve responsible and informed leadership that will ensure an equitable, community-driven, and child-centered education. We will accept nothing less” said Anna Shah, Dutchess County public school parent.
“The 10 NYS Board of Regents members who lacked the courage to vote against Governor Cuomo’s public school privatization agenda have now emboldened parents towards increased activism. Through the use of social media, traditional media and speakers forums parents will continue to inform and educate. They will forge ahead against these harmful policies using their best weapons…involvement in the political process (our eyes will on our legislators) and of course the 500,000 test refusals for Spring 2016,” said Lauren Isaacs Schimko, public school parent, Rockland County educator & Administrator of “Pencils DOWN Rockland County” on Facebook.
HV Alliance for Public education, is a grassroots organization dedicated to advocating for the rights of parents and public school children against harmful testing practices in the Hudson Valley. To join the Alliance or to learn more, please visit us here:
This arrived in my email box. I hope Regent Tilles, a member of the New York Board of Regents, reads it before he votes on the Governor’s flawed and punitive teacher evaluation program.
Dear Regent Tilles,
I am writing to you to share my own experience as a teacher in the hopes that you’ll see how absurd basing teacher evaluations on test scores can be.
Before I reveal my state growth score, let me give you some figures. After all, in any other job in the world, you are judged on performance. And Common Core is supposed to be about getting our students college and career ready, two places where they will be judged on performance. So after my principal gave me my state growth score, I decided to take a look at how my students actually performed on the state ELA test.
I had 31 students take the state ELA test. I am not going to count one student and his Level 1 score based on the fact that on the Part 1, he simply filled in A for each question in column 1 of the answer sheet, B for each question in column 2, C for each question in column 3, and D for each question in column 4 (that alone should be enough to see the flaws in having a test-based evaluation). 14 of my 30 students scored at Level 3. That’s 46.7%. 7 of my 30 scored at Level 4. That’s 23.3%. 70% of my students scored Level 3+4. The percents for the entire grade level in my building were 26% Level 3, 15% Level 4, 41% Level 3+4. In the regional scores given to us by BOCES, 42% of the students on my grade scored on Level 3+4. Other than the SED, anyone would look at my numbers and say not only did I do my job, I did it very well.
Here’s another way to look at the results. My students accounted for 27.5% of those who took the test on my grade level, but they accounted for 48.3% of our Level 3 scores and 43.8% of our Level 4 scores. Let me say that again. My students were a little more than 1/4 of all those on my grade level in my building who took the test and yet I accounted for nearly 1/2 the Level 3 and 4 scores.
Based on that data, the question I have is, “Did I do my job? Was I an effective teacher?” Well if I were a school myself, my percentage of students meeting state standards, 70%, would rank me 8th in all of the county.
So now the big reveal. What was my state growth score based on these fantastic results? A 20 out of 20? At least a 16 out of 20? No, I was a 1 out of 20. Yes, a 1. 70% of my students met state standards, yet based on the ridiculous growth formula SED uses that no one really understands, I was deemed to be an Ineffective teacher. Thankfully, my bosses recognize the job I do and due to my score on my observations and our local 20%, my overall score is enough to rate me Effective (barely).
I hope my situation shows just how deeply flawed a test-based teacher evaluation system is, and that you will do everything possible to make sure we eliminate it. I have no problem being evaluated. Contrary to what the governor may think, he did not create the teacher evaluation system. I have been evaluated every year during my 22-year career. All the governor and our legislators have done is create a system that simply DOES NOT WORK. I’m asking for your help in fixing it.
A proud teacher beginning his 23rd year
Can you believe this?
While everyone else is complaining that Governor Cuomo is crushing teachers with his punitive and research-less teacher evaluation plan, the New York Post complains that Governor Cuomo has capitulated to the teachers’ union by ordering a new review of the Common Core standards and assessments. Imagine that! The governor actually might have cared that 220,000 children opted out; he no doubt realized that 220,000 children might have 400,000 or so parents, and they vote. The New York Post seems unaware that in a democracy, it is usually a good idea to pay attention to mass movements.
The Post feels certain that Cuomo is kowtowing to those horrible teachers’ unions, always the enemy (the teachers’ union has now morphed into George Orwell’s Emmanuel Goldstein in “1984,” the quintessential enemy of the State).
Read the editorial. The Post will not be satisfied until there are mass firings of teachers.
Here are the closing lines:
We’ve said it before, and we’ll say it again: The unions and the politicians they control will make sure no system ever lets schools shed rotten teachers.
The only hope for kids is to flee these failure factories — to flee to charters or private schools, or out of New York altogether.
Oh, dear, where should families flee to?
Not to Connecticut; it has teachers’ unions.
Not to New Jersey; it has teachers’ unions.
Not to Massachusetts; it has teachers’ unions.
Not to Pennsylvania; it has teachers’ unions.
Flee, families, flee!
Flee to Tennessee! Flee to Mississippi! Flee to North Carolina! Flee to the Deep South! Flee to any state without a teachers’ union.
You won’t get better education but at least you can be sure that the teachers are without any representation.
Oh, and by the way, do the writers and workers at the New York Post belong to a union? Or is it a non-union shop?
Blogger Perdido Street School reports that 45 percent of the teacher ratings for the public schools of Buffalo, New York, were inaccurate. The ratings were outsourced to a company in Utah that acknowledged its errors. Its miscalculations resulted in 1,089 teachers receiving lower ratings than they should have.
The blogger quotes the story in the Buffalo News:
Lower-than-correct scores were given to educators who teach more than one grade level or subject and are required to meet multiple sets of student learning objectives. The company had rewritten its scoring calculations over the summer to enable it to produce scores more rapidly, Rosenthal said. But in doing so, it inadvertently created the calculation error for this group of teachers.
The district’s data chief, Genelle Morris, said a teacher brought the error to the district’s attention. According to the teacher’s manual calculations, she had met her performance targets, but that was not reflected in the online calculations produced by Truenorthlogic. The district checked her calculations and ran them internally through the district’s Information Technology Department and found it could not replicate Truenorthlogic’s scores.
The company soon uncovered the source of the error and the corrected results were posted online for teachers to view late Thursday morning.
Perdido Street adds this observation about the Governor who insisted on creating the evaluation system (APPR) and who insists it is “scientific” and “objective”:
It will be fun to see Cuomo twist himself into a pretzel to defend APPR even as the Common Core/Endless Testing edifice comes down around him.
Make no mistake – he will do just that.
APPR is his baby – his donors wanted it, he pushed for it, when pushback came, he fought to impose it on the state through the budget.
He will not want to admit it is as flawed and error-riddled as the Common Core implementation (which he can blame on NYSED), the Common Core tests (which he can blame on Pearson) or the Common Core curriculum (which he can blame on NYSED.)
When it comes to APPR, he has no one to blame but himself.
Mike Lillis, president of the Lakeland County Federation of Teachers, tells his members that education is at a “tipping point.”
“In my opening remarks last year, I painted a picture of education reform on the run, with evidence such as Inbloom shuttering its doors, 65,000 students refusing the state tests, and polls showing increased opposition from parents as well as educators to the Common Core standards and high-stakes tests.
“Well…a lot has happened since then. To borrow from Charles Dickens, who would have a great deal to say about our treatment of children today, in many ways what we are seeing now is the tale of two education systems; it is the best of times and the worst of times.
“Since I like to rip band aids off quickly, let’s talk about the worst of times first. Despite the fact that high-stakes tests took a beating last year, their use in teacher evaluation being discredited by the American Statistical Association, American Education Research Association, National Academy of Education, as well as parents and educators, this April the Legislature and Governor doubled-down by rewriting the law on teacher evaluation and dramatically increasing the role that tests will take on.
“Here is why I believe it is also the best of times. Within a couple of weeks of our legislative rout, the parents of over 220,000 students said, “enough is enough” and refused to have their children take the state tests. This level of direct parent participation is unlike anything we have seen before in New York and no, it is not because the “union told them to.”
“This is a grassroots movement of parents that have tried every other means they have to protect the integrity of their child’s education.
“There are not 220,000 parents in New York that wake up daily and try to figure out which act of civil disobedience is best for them that day. This is a movement that will grow until sanity is restored to our classrooms. They have built a lever, and it will continue to grow until it is successful.”
He notes that Commissioner MaryEllen Elia said teachers were”unethical” if they encouraged opt outs.
“Since the Commissioner would like to discuss ethics, I would like to contribute some of my own thoughts about State Education Department policies and hopefully begin a debate about whether or not they are ethical or even moral.
“I do not believe that it is ethical to correlate proficiency on grade 3-8 math and ELA assessments to a score of 1630 on the SAT, a score that only 34% of college bound students achieve nationally.
“I do not believe it is ethical to label students. More importantly, I do not believe it is ethical for the state of New York to label students “College and Career Ready” based on a single annual test.
“I do not believe it is ethical to take students that remain on target to getting a 1500 on the SAT and labeling them not ready for college or a career, in third grade, fourth grade, fifth grade, and on and on through high school. I believe this is state-sponsored abuse.
“I do not believe it is ethical to use a formula to evaluate the state’s 3-8th grade teachers that guarantees 7% of them will be ineffective before a single student takes the tests.
“I do not believe it is ethical to put a test ( a test I remind you that statistically is designed to have only the top 34% college bound students nationally pass it) and have students know that their performance will have a very real impact on the career of someone they love—their teacher.”
Lillis concluded: we are a tipping point. Which way will we tip?