Stephen Sawchuck notes in his blog at Education Week that a pattern is emerging from teacher evaluation programs: The highest ratings go disproportionately to teachers of advantaged students and the lowest ratings to teachers of students who are disadvantaged. He wonders whether this suggests that the ratings systems are biased against those who teach the neediest students or does it suggest that the schools with high numbers of disadvantaged students get the worst teachers.
I am reminded of the joint statement released a few years ago by the American Educational Research Association and the National Academy of Education, which predicted that those who taught the neediest students would get the lowest ratings because of factors beyond their control. Their schools are apt to get less resources than they need and have larger classes than is beneficial to students. It may have fewer science labs and computers. Its students are likelier to be ill and have a higher absentee rate because of inadequate medical care.
That report found that:
Even when the model includes controls for prior achievement and student demographic variables, teachers are advantaged or disadvantaged based on the students they teach. Several studies have shown this by conducting tests which look at a teacher’s “effects” on their students in grade levels before or after the grade level in which he or she teaches them. Logically, for example, 5th grade teachers can’t influence their teachers’ 3rd grade test scores. So a VAM that identifies teachers’ true effects should show no effect of 5th grade teachers on their students’ 3rd grade test scores two years earlier. But studies that have looked at this have shown large “effects” – which suggest that students have at least as much bearing on the value-added measure as the teachers who actually teach them in a given year.
One study that found considerable instability in teachers’ value-added scores from class to class and year to year examined changes in student characteristics associated with the changes in teacher ratings. After controlling for prior test scores of students and student characteristics, the study still found significant correlations between teachers’ ratings and their students’ race/ethnicity, income, language background, and parent education. Figure 2 illustrates this finding for an experienced English teacher in the study whose rating went from the very lowest category in one year to the very highest category the next year (a jump from the 1st to the 10th decile). In the second year, this teacher had many fewer English learners, Hispanic students, and low-income students, and more students with well-educated parents than in the first year.
This variability raises concerns that use of such ratings for evaluating teachers could create disincentives for teachers to serve high-need students. This could inadvertently reinforce current inequalities, as teachers with options would be well-advised to avoid classrooms or schools serving such students, or to seek to prevent such students from being placed in their classes.
So, do schools serving low-income students get worse teachers, or do teachers in low-income schools get smaller gains because it is harder to succeed when kids do not have the extra resources they need and are burdened with poverty? I would say it is some of both. For one thing, brand new teachers are disproportionately placed in low-income schools, some having just finished their teacher training, as well as TFA recruits who have only 5 weeks of training. First-year teachers are likely to be less successful than experienced teachers. At the same time, it is harder to get big test score gains in schools where there are large numbers of students who don’t speak English and who have high needs.
Cross posted at
http://www.opednews.com/Quicklink/Stephen-Sawchuck-Asks-Do-in-Best_Web_OpEds-Control_Diane-Ravitch_Education_Evaluation-140831-311.html#comment509237
with this comment
I hope that if you read the links to the truth about what it takes to teach and what is critical for LEARNING to occur, that you send others to the links now that you know the conspiracy to end public education is powerful enough to control the media. if our citizens can be sold snake oil as reform when all real evidence says otherwise then nothing will change and the road to opportunity in America is shut down. This essay is important.
Well Diane, thank you (I think?) for coming to the conclusion that I’m not the ‘worse teacher’ after 20 years in high-poverty Title I schools.
I rather think I am one of the better teachers myself, but that could be pride I suppose. Well that and the fact that I hold a BA, 2 MAs, completed an ABD doctoral program, became National Board Certified, and won 2 district teacher of the year awards.
Your argument and Mr. Sawchuck’s arguments both seem to be based on what standardized test scores say is ‘success’ and I would argue that there is far more that we ‘worse teachers’ in high-poverty schools teach and do that isn’t covered by any existing standardized measure.
I’ve said this here before and I’ll say it again: for years I have seen ‘good teachers’ from high income schools come into our school to show us how it should be done. Nearly all leave within a few weeks, often angry or in tears, because they can’t work with children who don’t come to school already reading and fluent in writing, mathematics, science and social studies and whose parents don’t spend money on supplies, trips, and often don’t even have food or electricity in their homes, let alone food or medicine.
They have no frame of reference for dealing with children whose parents are jailed and who are being raised by relatives or in foster care, children who are sick, in need of glasses and dental work, often angry and hostile and acting out throughout the school day. They start saying that ‘those children’ are not educable, showing their truly low expectations.
My colleagues and I, however, come back year after year and manage to deal with all of this and not having needed materials, lacking computers, etc. just as you say, yet we manage to teacher these children not to hurt each other constantly, how to read, write, do math, science, social studies, music, art, etc. with materials we purchase with our own money. and we actually laugh and have fun more often than not.
Yes, we are under attack by VAM and Danielson/Marzano and lies like the 90/90/90 snake oil nonsense. Yes, we will be the first to go. Yes, we are good, dedicated, loving, educated professionals who do see many inexperienced colleagues come and go year after year because what we do is damn hard and we get no respect.
Chris in Florida,
I will be the first to say that I could not possibly do what you do every day. That’s why I respect teachers, because you are our first responders in educating and protecting our children; you have a tough job, and I would never second-guess you. If you told me how to write and edit, we would disagree. But you will never find me telling teachers how to teach.
Chris in Florida: much appreciation for all you do.
Never forget the mock-Latin aphorism:
“Illegitimi non carborundum” = “Don’t let the bastards get you down.”
😎
Few really have any comprehension of the challenges faced by both students from underprivileged homes or the teachers who have them in classes. The absence of passing along the basics of life that is a staple of middle class America leaves poverty kids lacking the essentials to be able to grasp the concepts being learned in the class.
Two separate pieces which provide some idea of the challenges faced by poverty students are:
http://online.wsj.com/news/articles/SB998511295703108758
If we fail to provide extra resources to bring poverty students up to the point of equilibrium with middle class students, we are implementing fatally flawed public policy and one penalty is that our society will continue to bear the burden of enormous criminal justice system expenses, including prison for those who get caught committing felonies.
And when we say resources, let’s be clear what those resources are: extra materials, health and social-emotional support staff, community resource staff who are trusted by the community and can make necessary connections, etc.
My district is providing additional resources to high-needs schools. Some small portion of these are resources as above; the vast majority of money, though, is going to more professional development, more coaches, and more data analysts. As an experienced veteran teacher, I am not supported by PD on basic classroom management and teaching strategies – actually, they are a waste of time and energy, keep me from more useful PD or making connections with students, and are pretty offensive, given that I have been extremely successful in the classroom and am called upon to host observers and new teachers regularly.
However, my district speaks glowingly of its commitment to equity via these “supports”. I think we need to make that harder for them to do.
I agree with you E. Rat.
When I told an ex-principal of mine that I deserved to have professional development on my own level of expertise instead of having to sit through a “how to begin doing _____ ” workshop that I’ve already been made to attend over a dozen times (and the workshop is based upon work I did a decade ago) she said I wasn’t being a ‘team player’.
Far too many administrators, coaches, data catchers, observers, etc. and not near enough social workers (we share one with 2 other schools), psychologists (we share one with 3 other schools and all their time is spent on data crap instead of helping students), and nurses (we share one with 2 other schools).
Instead of creating yet another layer of bureaucracy and useless paper pushers in the endless data loop that money needs to be invested in social services for our needy, sick, and hurting children.
I’m so sick of being told what and how to do things by people who have NEVER in their career had to do those things themselves and who have absolutely NO real world experience in what they are supposedly ‘teaching’ us teachers how to do.
AMEN! It is the same story across America. If doctors were subjected to this kind of ‘support’ the hospitals would fail, too.
“Actually, they are a waste of time and energy, keep me from more useful PD or making connections with students, and are pretty offensive, given that I have been extremely successful in the classroom and am called upon to host observers and new teachers regularly.”
WHAT!?!?!?
You don’t appreciate being “professionally developed”????
I’d love to be able to say that things are fair and just but it isn’t so. First of all the Dr. Marzano tool for evaluation and the methods of teaching I’m supposed to follow from it are not what I learned in my classes for students with severe disablities. This year in addition to all the Design Questions from Dr. Marzano’s mat, I am to facilitate higher level thinking skill activities or depth of understanding for my students, the lowest one percentile students. Common Core is on line for math and Language Arts and we are supposed to look at the grade level standards yet teach the student on his or her cognitive level. I’m detecting a contradiction here. The kicker is that the students, yes take an alternate test, yet are tested on grade level content that they weren’t able to master due to not having the pre skills, memory, sensory processing…. We are to develop stratagies to help raise our test scores. It feels like it is designed for the students and teacher to fail. I wish I could cure my students of their disabilities. I also wish politicians and the media spin doctors could come visit. My students can and do learn. It’s on their own schedule in their own way that may not test.
Strong correlation should lead to a good discussion of causes. But 90% of my students are economically disadvantaged. Teen pregnancy, health, crime, autism, truancy,, family turmoil – and still the “experts” on Mount Olympus at our capital tell me I’m ineffective and need to try harder. Spew them. They are out of touch with reality and part of the problem. Last year, one of my students was going to school and working two jobs. His mother found out she was terminal and he was supporting the family, including a sick aunt and grandfather. Another student was trying to leave the gang life and found out he had a daughter on the way. He wanted to be a good father and finish school. We started one tutoring session on fractions and finished with fatherhood.
The clueless tea party Republicans that run our state sit in their marbled halls, well fed and pompous. They despise teachers and do everything they can to make our jobs difficult. I have resigned myself that as long as I continue to teach this population, the politicians will eventually end my career – unless voters end their’s first.
I have recently retired from a career I loved. My Masters’ was in Gifted Ed., my personality and demeanor found sixth grade students needing a low key, humorous male figure in my class as well, several with IEPs or 504 plans…my teammates were similarly talented and compassionate, one with National Board certification. It was a real juggling act to reach AYP and value added each year…oh, the eighty plus years of combined classroom experience between the three of us enabled us to manage–with a whole bunch of early mornings and long afternoons. I now sub, and love teaching a class of kids without Damocles’ Sword (swords, actually!) hanging over my head….
Your story is my story. We had a wonderful small middle school where we worked as a team. We were at the top of NYC on all measures of excellence and our kids were accepted at the top schools… my curricula became famous and I was chosen by Harveard and Pew as a cohort for the real National Standards research. Within two years I was harassed out, accused of all manner of things, and finally of incompetence, even s I was the NYSEC Educator of Excellence. In NYC al the best veterans experienced this, and all the schools that worked well, like mine, were ended when they left. Relegating great teachers to become substitutes is the final blow.
The process has been imported from NYC. If it works, destroy it.
https://vimeo.com/4199476
I wrote this years ago:
http://www.speakingasateacher.com/SPEAKING_AS_A_TEACHER/The_Insane_War_on_Teachers_and_Democracy.html
Teachers in this country are being subjected to a ridiculous evaluation system based on voodoo math. If you thought the statistical impossibility of all students being “proficient” under NCLB wasn’t disheartening enough,VAM continues the absurdity by punishing the teachers of learning impaired, ESL/ELL, children of poverty, mental illness, trauma and general family dysfunction. If we can just ignore all these variables and blame teachers, we can solve the problems? More assessment is not going to solve the problems! If we continue to scapegoat teachers, who will enter this field and teach our most vulnerable children? We need to stop politicizing education, and let teachers do their job.
VAM scores track socioeconomic status. School report cards track socioeconomic status. SAT and ACT scores track socioeconomic status. Gee, who would of thought?
In my school, years back, a student came into my 7th grade reading at the preprimer level. He struggled for a couple of months because, in 6 year of school he had gained nothing. However, after 2 months, the light went off and he gained 4 years in reading.
Most reasonable people would call the student, teacher and school a success. However, at the 4th grade level in reading, he was still not near proficiency. No matter how much value added or “average” yearly progress and more nonsense or tweaking, they are all considered a failure.
While our school was challenged to bring that student up 7 grade levels, the burbs only needed to bring them up one. Why? Because they look at averages of students as commodities, not individual gains. The real question is how many students gained under your watch.
Details and more examples in my upcoming book, Brainstorming Common Core. It’s heading to the publishe at the end of this month
This thread is by far one of the most painful threads I have read. I know how hard teachers work, how much time spent to reach out to a struggling student and now being thrown to the discard pile along with their students. I listened to a TED talk yesterday about an F and a smiley. The smiley was for two right answers. If only teachers could be scored on the positives, i.e., the student who finally achieves an “aha” moment, the students who step up to learn despite an overwhelming array of responsibilities.
Thank you to everyone who is trying to make it possible for teachers and students to thrive and grow despite overwhelming barriers and “negativeness”.
Reblogged this on Teacher Talk and commented:
Evaluations are deeply affected by factors beyond the teacher’s control. As a teacher of students who consistently earn low test scores, VAM and test-based teacher evaluations have always concerned me. Thankfully, Vermont doesn’t evaluate teachers via test scores, and hopefully they never will.
Reblogged this on peakmemory and commented:
The essential failure of value added measures of teaching:
“Even when the model includes controls for prior achievement and student demographic variables, teachers are advantaged or disadvantaged based on the students they teach.”
This was old news in 1985, but Ed Week and its ignorant “reporters” are always reinventing old wheels and rediscovering reality. Is this because none of them knows anything about teaching? A little personal history.
By the mid-1980s, I had decided to remain in Chicago’s general high schools (usually all minority inner city) as an English teacher. This was despite the fact that I had a BA from the University of Chicago (’69), a couple of years of law school (DePaul, no degree) and a lot of experiences beyond teaching. The reason was that we had to promote inner city teaching, and work with our brothers and sisters at those schools.
By the mid-1980s, the main conversation was whether inner city teachers should receive additional pay because of the realities we would face. I could list dozens of them, heart rending realities, but will simply put this forward:
Despite the fact that anyone who taught within a hundred miles of Chicago would say they were “Chicago teacher” when we were at conventions (say, NCTE in 1994), I always had two questions for these “Chicago” teachers. First, where do you teach. (Oak Park River Forest and New Trier are not in “Chicago”!!!). Second, tell me the names of the dead kids you knew.
This second challenge was always the key.
Everyone who has taught more than two years in the real inner city has a list in our heads, marching like Banquo’s ghosts, of dead kids. Some years (1997 – 1998), they pile up in huge numbers. Other years it’s “only” one or two. But nobody who has taught, principaled or coached in the real inner city lives without that haunting dead kid list. And it was a central metric in our lives. One Monday morning you would arrive for your U.S. Lit class and the empty desk would be “Flaco” (there are hundreds of these, as anyone who knows drug gangs knows) who was shot to death at Leland and Kedzie Saturday night in a confrontation between the Insane Unknowns and the Latin Kings. Etc.
If you lasted in the real inner city long enough, the list got so long you forgot some kids on it. (I eventually wrote it down after watching Antwan Jordan die with a bullet straight through his head in December 1997 outside Bowen High School and then calling in on the walkie talkie I carried those years the “187” after the paramedics confirmed what was obvious…).
When the first “metrics” came out against us in the mid 1990s at the birth of mayoral control and CEO-based “school reform” in Chicago, I couldn’t believe they were going to try and get away with that bullshit. But of course we all know now that Paul Vallas, the first large urban district CEO (appointed by Mayor Richard M. Daley in July 1995) was just the beginning of a tsunami of garbage, MBA and “data driven” variety.
And now it’s September 2014, 20 years after the first lies were floated and the first “metrics” spewed out by the ruling class. And on August 27, 2014, I spoke at the meeting of the Chicago Board of Education about how the latest “performance policy” for the nation’s third largest school system was, once again, discriminating against schools and teachers who served the poorest kids. I only cited one of the dozen or so “metrics” utilized this years in Chicago against the inner city high schools — AP scores.
I taught both Advanced Placement English courses (Literature; Language) during the years before I was fired and blacklisted by Vallas and the Daley school board. And the best I would be able to tell the kids who volunteered for those electives was that they might — MIGHT — get a three on the AP exam in May if they did everything I told them to do beginning the previous July. But fact was, the fours and fives had already been siphoned away into the “magnet” high schools. The kids “left behind” in schools like Amundsen and Bowen were behind for the same reasons they knew more dead kids than their suburban or magnet school peers. And despite all that “all children can learn” nonsense, they were not going to “catch up” AP English style in one year.
But I also told them that once they had completed my class (which read all — ALL — of Anna Karenina during the annual “long March” they would be ready to handle any first year English class at any university or college in the USA — including my own alma mater, the University of Chicago. And year after year, until Paul G. Vallas and the Chicago Board of Education voted at their August 2000 meeting to fire me, the “kids” came back and told their younger generation that I was right.
I won’t go on.
The data driven maniacs currently still in power are seeing their power waning, but they are going to have to be destroyed, not discussed with. They have been exercising ruthless and ignorant power for a generation, and they won’t be dislodged easily. But as I’m reporting today, at least the Resistance is on the way.
Unless one has spent significant time at a high poverty school, one does not know of what one speaks.
VAM takes none of the characteristics of high poverty schools into consideration. Attendance rates lower than 95%, low reading skills (see Cap Lee’s comment above), lack of literacy in the home, English as a second language, high mobility rate, often, no transportation and no phone for parents, parents and kids in survival mode (see Maslow’s hierarchy of needs).
Our profession has been usurped by charlatans.
Yes, but worse than charlatans, these people are intent on destroying the road to opportunity for Americans, and ending the sharing of real history an thus ending democracy.
Click to access hirsch.pdf
I am not a fan of E.D. Hirsch. He is part of the failed schools narrative.
New York Times op-ed by Joe Nocera makes the point that absolutely no other industrialized country has a teacher accountability system.
Also, the other high performing countries’ schools had higher compensation for teachers but had lower per capita costs than does the US.
Note: this is a copyrighted article of The New York Times.
dl
Opinion
OP-ED COLUMNIST
Imagining Successful Schools
AUGUST 29, 2014
Joe Nocera
What should teacher accountability look like?
We know what the current system of accountability looks like, and it’s not pretty. Ever since the passage of No Child Left Behind 12 years ago, teachers have been judged, far too simplistically, based on standardized tests given to their students — tests, as Marc S. Tucker points out in a new report, Fixing Our National Accountability System, that are used to decide which teachers should get to keep their jobs and which should be fired. This system has infuriated and shamed teachers, and is a lot of the reason that teacher turnover is so high, causing even many of the best teachers to abandon the ranks.
All of which might be worth it if this form of accountability truly meant that public school students were getting a better education. But, writes Tucker, “There is no evidence that it is contributing anything to improved student performance.” Meanwhile, he adds, test-based accountability is “doing untold damage to the profession of teaching.”
Tucker is one of the grand old men of education policy. In the 1970s, he worked at the National Institute of Education, followed by a stint at the Carnegie Corporation. In 1988, he founded the National Center on Education and the Economy, whose premise, he told me recently, is that, in order to meet the demands of a global economy, our educational system needs to be re-engineered for much higher performance.
Not long after founding the N.C.E.E., Tucker began taking a close look at countries and cities that were re-engineering successfully. What he came away with were two insights. First was a profound appreciation for the fact that most of the countries with the best educational results used the same set of techniques to get there. And, second, that the American reform methods were used nowhere else in the world. “No other country believes that you can get to a high quality educational system simply by instituting an accountability system,” he says. “We are entirely on the wrong track.” His cri de coeur has been that Americans should look to what works, instead of clinging to what doesn’t.
The main thing that works is treating teaching as a profession, and teachers as professionals. That means that teachers are as well paid as other professionals, that they have a career ladder, that they go to elite schools where they learn their craft, and that they are among the top quartile of college graduates instead of the bottom quartile. When I suggested that American cities couldn’t afford to pay teachers the way we pay engineers or lawyers, Tucker scoffed. With rare exception, he said, the cost per pupil in the places with the best educational systems is less than the American system, even though their teachers are far better paid. “They are not spending more money; they are spending money differently,” he said.
Tucker would not abolish tests, but he would have fewer of them. And they would have a different purpose: In the high-performing countries, the tests exist to hold the students accountable, rather than the teachers. Meanwhile, he writes, “in most of these countries, the primary form of accountability for the school and its staff is high-profile publication of the average scores for the exams for each school, often front-page news.”
When a school falls short, instead of looking to fire teachers, the high-performing countries “use the data to decide which schools will receive visits from teams of expert school inspectors. These inspectors are highly regarded educators.”
Tucker envisions the same kind of accountability for teachers as exists for, say, lawyers in a firm — where it is peers holding each other accountable rather than some outside force. People who don’t pull their own weight are asked to leave. The ethos is that people help each other to become better for the good of the firm. Those who successfully rise through the ranks are rewarded with higher pay and status.
Would the teachers’ unions go along with such a scheme? The unions would certainly have to shed some of the things they now have, such as control of work rules. But they would gain so much else: “Management would get their prerogatives back and would be held accountable for results, but the professionals, granted far more autonomy, would be also holding each other accountable for the quality of their work, as professionals everywhere do.”
As our conversation was coming to an end, Tucker told me that he was working with the State of Kentucky to implement some of the reforms he had outlined in his report. If it works there — and there is no reason it shouldn’t — perhaps we’ll finally get over our fixation with test-based accountability, and finally re-engineer our educational system the way every other successful country has.
The Conference Board has just released a summary of its in-depth research which indicates severe labor shortages in several areas as the number of retirees is projected to exceed the number of qualified new entrants into these fields.
https://www.conference-board.org/press/pressdetail.cfm?pressid=5266
These projected labor shortages may well provide the public policy motivation to finally utilize the enormous talent that has traditionally been allowed to lie fallow in this country:
the high achieving, low income students. These individuals were found in the Brookings Institution Hamilton Project to frequently choose colleges that are a poor fit for their needs and abilities, with the result that their contributions to society frequently end up being subpar as compared to their potential had they attended a more competitive college with superior resources.