Marcus Winters recommends using value-added assessment to get rid of “ineffective” teachers. His paper was published by the conservative Manhattan Institute, which regularly issues his and others’ critiques of unions, tenure, seniority and any kind of job protection for teachers.
Many studies–and practical experience–have demonstrated that value-added assessment is unstable, unreliable and inaccurate. A teacher with a high rating may have a low rating the next year. The National Academy of Education and the American Educational Research Association published a joint paper warning that VAA says more about which students are assigned to the class than about teacher quality.
And then there is the problem that there is no district that has been able to demonstrate that VAA actually identifies ineffective or effective teachers. When New York City published its value-added ratings, the allegedly “worst” teacher in the city taught immigrant students who cycled in and out of her class as they learned English. As Linda Darling-Hammond and others have warned, value-added assessment will encourage teachers to shun the students with the highest needs and gifted students. Neither will produce the expected gains.
It is interesting and curious that Arne Duncan’s favorite innovation happens to be the favorite solution of the right to find and fire “ineffective” teachers.
I will never understand why the rightwingers are so devoted to high-stakes testing, which is known to produce narrowing of the curriculum, teaching to the test, gaming the system, and cheating. What’s to like? Maybe they like it because it gives them a club with which to bash teachers, their unions, and public education.
Diane Ravitch's blog 
There’s another reason: Paring down to testing leaves available the money that was used for other aspects of education, money that can then be swooped up by “entrepreneurs” who recognize that budgets will not get reduced, simply redirected.
Follow the money directly to Pearson.
Do true conservatives love testing? Or perhaps the question should be Do true conservatives love government mandated testing? I find it odd that most who proclaim to be conservative don’t understand that it’s not conservative to increase the size, power and scope of govt by using it to mandate testing…not to mention spending the money to do it. Perhpaps it’s because they aren’t really conservative and are just answering to the corporate interests who have confused being conservative with being greedy. These same neo-conservatives are the ones who passionately stand up to the govt trying to “control our bodies” through Obamacare, yet demand that our govt get into our kids’ minds. Do they ever stop and think how they contradict themselves?
These neo-conservatives are neither neo nor conservative but very regressive in their thinking/being/doing.
They may love it because it makes the complicated simple. Whittling a child’s life and the complex process of teaching him or her down to a single score is easy. Like the bottom line that is chased fervently in the business world. Schools are not businesses. Our bottom line is more complex.
I debate often with a friend’s mother about education issues. She can not accept that there is any objective way to ascertain a student’s knowledge and skills than through a standardized test. When I suggest that teachers are trained to do this, just like doctors are trained to diagnose illness, or accountants are trained to identify errors in the books, she balks at allowing mere teachers to do this. I think this is the same issue that the reformers have. They want to keep all the control and the only tool that they understand is the test. Like the patient who would rather get the 4th and 5th opinion rather than accept the diagnosis, the reformers don’t want to listen to the experts. They don’t want to listen to the research. They just want to keep using the stethoscope to listen to their strong heartbeat while ignored all the other tools that diagnosed the cancer.
In my experience teacher’s evaluation of learning, at least as reflected through class grades, are not a good way to measure student’s knowledge or skills.
Don’t know what experience that is, but in MY experience, part of the artistry of teaching is in knowing each student, and in being able to evaluate each student’s progress via multiple means, and to condense that overall impression into a grade. Those grades say an enormous amount, and are a way more instructive, informative, and useful way to determine how a student is doing, in terms of skills, knowledge, and overall interest in learning.
TE,
Allow my broken record of a mind to remind you that it is logically and rationaly impossible to “measure” any student’s knowledge or skills. It cannot be done without error. So that leaves us to the teacher’s judgement, nothing more and nothing less and it will always be less than “scientifically” accurate. And really the best way to evaluate/assess a student’s learning would be through a portfolio process in conjunction with the student evaluating his/her own learning. I still haven’t figured out who declared him/herself god and determined that we have to sort and separate out students with the logical atrocities that are grades in whatever format they take.
Come join the reading and discussion of the problems of educational standards, standardized testing and grades in general at my blog “Promoting Just Education for All” @ revivingwilson.org .
Duane
I certainly agree that grades are based on many things, but some of those things have little to do with skills, knowledge, and overall interest in learning.
In the case of my son, the B in his pre-calculus class was a much less accurate assessment of his skills, knowledge, and interest than the score on the standardized math exam he took.
Tennessee teachers have an annual evaluation that is based 50% on observations (TAP model) 15% achievement (state test or other approved instrument) and 35% value added. Every professionally licensed educator and administrator in a school district has a value added component in their annual evaluation. The process was implemented in 2011-2012 and is problematic for all. The background paper included in this post is very helpful in understanding why TN should review and revise its current system.
barbide,
If you are a teacher in TN you have my condolences.
Duane
I read somewhere recently that skiing is highly correlated with student achievement on standardized tests (i.e., students who go skiing score higher). Therefore, I propose a value added assessment of ski resorts. We could organize ski trips for, say, Chicago Public School students to a variety of different resorts. Then we could look at the value added scores on standardized tests by resort. Resorts which raise student scores would be eligible for bonuses and incentives, while resorts which keep scores the same or lower would be “prioritized” for “turn-around”, in which the entire staff of the resort, from ski instructors to janitors, would be fired and replaced with cheaper staff and the whole resort would be put under new management.
Either that or we could just evaluate teachers by throwing lists of teacher names down the stairs and firing all of those whose names land on the bottom step.
“Either that or we could just evaluate teachers by throwing lists of teacher names down the stairs and firing all of those whose names land on the bottom step.”
I thought that was what we were doing now!
Pretty much.
“I read somewhere recently that skiing is highly correlated with student achievement on standardized tests (i.e., students who go skiing score higher).”
Of course that wouldn’t have anything to do with the fact that skiing is a fairly expensive hobby to have/sport to do since standardized test scores don’t have anything correlation to SES status of the student’s family.
That would be the point, yes.
The word “conservatiove” doesn’t really fit the orientation of mind we are talking about here. There are very few of our highly complex and richly-textured traditional values that the people in question exhibit anything beyond a purely token interest in conserving, indeed, perhaps only those values that can be reduced to monetary terms. That may give us a clue to the syndrome. We note a preference for one-dimensional value systems, and a short-sighted tendency to see means — like money and test scores — as ends in themselves.
I haven’t found the best word to describe this condition yet, but in this context I think “Value-Addled” might just fit.
And interestingly enough, someone who wishes to conserve tradition would want to go back to a time when teachers were trusted to make judgments about their students, were respected for their talents beyond simply teaching reading and arithmentic, and were protected from criticism of know-nothings outside of the profession.
It’s the new apartheid. It’s educational. If you limit education (by testing and narrowing the curriculum) and opportunities you limit employment and life opportunities. It’s the new “haves” and “have nots”. I have an education and you don’t. My life will be better and yours won’t. It’s the new class system coming to a school near you.
Conservatives may love high-stakes-testing/teacher-discharge because it weakens teacher unions. However, this does not explain why Democratic mayors, governors, and national Democrats (including Duncan/Obama) love high-stakes-testing/teacher-discharge.
The most likely explanation for the rapid spread of high-stakes-testing/teacher-discharge is that — on superficial analysis — it rationally addresses the real problem of identifying/removing poorly-performing teachers.
Under the traditional principal-observation evaluation system, very few, if any, teachers are discharged. Common experience suggests that, if a school system discharges fewer than 1% of teachers for poor performance each year, the school system is retaining many poorly-performing teachers. Virtually all parents — recalling their own years as students and thinking about their children’s teachers — can recall at least one poorly-performing teacher. In these circumstances, school reform proposals that focus on identifying/removing poorly-performing teachers will make sense to parents/voters.
High-stakes-testing seems like a rational way to evaluate teachers. The teacher’s job is to educate the students; it makes sense to measure a teacher’s effectiveness by how much the teacher’s students learn during the year; standardized tests measure how much students have learned; value-added models control for differences in students’ abilities.
This analysis — albeit superficial — makes perfect sense (indeed is compelling) to most parents/voters, particularly parents/voters who themselves attended suburban and/or private schools and who have little/no first-hand knowledge of life in the low-SES-area/inner-city public schools.
Of course, this superficial analysis is wrong — horribly wrong in the case of the low-SES-area/inner-city schools that are the schools actually causing the cry for school reform. Unfortunately, explaining why this superficial analysis is wrong is complicated/difficult, particularly when speaking to parents/voters unfamiliar with life in the low-SES-area/inner-city public schools.
Although there are no doubt some poorly-performing teachers in the low-SES-area schools, teacher quality in these schools is a minor problem compared to the huge social problems and the school-based problems of minor but endemic student misconduct and students reading far below grade level. Discharging a few teachers will not help these problems.
More importantly, in the low-SES-area schools, there are so many “problem” students and so much student turnover that high-stakes-testing cannot reliably measure teacher quality — a teacher’s test scores will be largely driven by the student mix and, in the low-SES-area schools, the student mix will be widely different from class to class and school to school. And, of course, the high-stakes-testing has tremendous adverse side effects — teaching to the test, narrowing the curriculm, encouraging cheating, discouraging teacher cooperation, and discouraging teachers from teaching “problem” students.
But — explaining all this is hard work; most voters/parents just don’t have the time to listen. And, unless the voter/parent personally knows life in the inner-city school, much of the explanation is unconvincing.
That’s why the corporate school reformers are winning the battle. .
Diane,
I’d love to see you debate Marcus Winters on the appropriate use of value-added modeling. Would you consider that?
Ken
Ken,
Who is Marcus Winters? What are his thoughts? Link please!
Thanks,
Duane
He is a very strong supporter of vouchers and opponent of teachers unions and public education
U of Arkansas
Here you go:
http://www.manhattan-institute.org/html/winters.htm
The use of value-added measures in NYC was a travesty.
Gary Rubinstein analsyzed the data and found that they were utterly meaningless.
Ken,
Thanks. Will have to read.
Duane
Sorry, I didn’t see the link at the top of the article-DOH!!! But thanks for the link, will attack it later as, boy, the first sentence of the executive summary is a doozy and patently false for a number of reasons-“Public school teachers in the United States are famously difficult to dismiss.” Wow, and not in a good sense, what a way to start.
Marcus Winters is one of those researchers on the right who doesn’t like public schools, unions, or teachers who have any rights
Diane Ravitch
Khirsh,
The last time you appeared I asked that you identify your position. Do you work in a school? Are you a teacher? Adminstrator? Charter manager? You always seem to fade away when challenged even though you attempt to challenge Diane. Man up!
Who are you addressing?
Diane
Sorry, I see you addressed Ken Hirsh. He can answer for himself. He does not work in a school. I think he manages money and supports charters. Ken?
Diane Ravitch
This is old but close enough:
http://gothamschools.org/author/ken-hirsh/
I think it would be great to have public debates amongst the most prominent advocates/researchers/etc on various sides of the major issues. Diane certainly fits the bill! When I read her attacks on the viewpoints of others (nothing wrong with that!), I wish I could see a realtime debate on the issues.
Ken, given the fact that the people on the “other” side have a near-monopoly on the media, I think it would be refreshing to have the opportunity to debate Michelle Rhee or anyone else.
The pro-charter, pro-privatization, pro-testing crowd has not only Rhee and Klein, but Arne Duncan, Bill Gates, Education Nation, David Brooks, etc.
Thanks Diane. I agree that there are very few well-known people that aren’t formally affiliated with the unions that take the union side. You are the most prominent person that I can think of.
Ken, I have never been a member of any union. But unions protect working people from exploitation. And they created a middle class in this country. We play with social dynamite by destroying them. Diane
When you were in favor of VAA, what was your rationale? What made you change your view? That might provide a guidepost as to what it will take to change people’s minds, Also, absent a competing assessment model–VAA is what we are arguing about. The answer it seems, is the development of a competing model that factors in the issues that generate the most concern.
California is one of the few states where the progress of VAM in teacher evaluation has been slowed down. Still, it’s not like the issue is going away. In this blog post, I lay out arguments to suggest that, at a bare minimum, we should table the conversation for a while:
http://accomplishedcaliforniateachers.wordpress.com/2012/09/14/after-ab5/
In any con game, there are reasons why the operator is selling the con, and there are reasons why the mark is buying the con.
I think we have to be careful to distinguish the two subpopulations involved in the game.
Exactly. And our goal is to expose the former to the latter.
Not an easy thing to do.
A really effective hook can be extremely difficult to extract from a sucker’s mouth.
My husband marked a column by Eric Zorn in the Chicago Tribune this morning. I could hardly wait to get on the computer to tell you about “Why Teachers Have Test Anxiety.” He got it right! He listened!
Found this..a little off topic but still informative.
Why people look down on teachers
By Corey Robin
http://wapo.st/O3GNLg
I’ve spent the last few days trying to figure out why people — particularly liberals and pseudo-liberals in the chattering classes — hate teachers unions.
Yay! He and I exchanged emails
Diane Ravitch
Please send a link to Zorn article
Diane Ravitch
Eric Zorn article
http://blogs.chicagotribune.com/news_columnists_ezorn/2012/09/test-anxiety.html
Eric Zorn read the research! Good for him
Diane Ravitch
Diane – yes, he read it, but many of his commenters still seem to need help understanding it. I still think a published dialogue (print or video) between the two of you could help. He has a way of asking probing but relatively fair questions, and you have an eloquent way of explaining things simply enough for most average people to understand.
Because a populace capable of higher order thinking is harder to control.
As a lifelong “rightwinger,” I disagree with you that we love high stakes testing. You are lumping all sorts of folks together and calling us “right wing” or “conservatives” and I do not think our idea of proper testing is “high stakes.” What most of us are is “traditionalists.” We do not believe group projects or student game inventions are often a wise use of class time, nor dowe believe most students can “discover” their own knowledge. We do not want the teachers to be boring drones but we do wasn’t the teachers to be in charge—not “guides on the side.” We believe that if one of us, Michelle Rhee, Jeb Bush and Joel Klein were 6th graders in a collaborative group, there is practically no chance in the world we could ever discover the Pythagorean Theorem or figure out the best method for long division or write a poem of the quality of John Donne or a novel comparable to Jane Eyre or a play such as one of Shakespeare’s or even Wilder’s Our Town.
What the “right wingers” I know want in the way of testing is an assessment that shows what a student knows and can do so that the teachers can better remediate his deficits and encourage his strong points and interests. The results of the tests for a whole classroom as well as for a whole grade of students and then for the whole school system should be used to analyze the classroom materials and methods of instruction to discern which are working satisfactorily and which should be replaced. The results of the tests can and should give clues as to strengths and weaknesses in teachers’ performance and content knowledge so that professional development will be appropriate and unfit teachers can be removed if they cannot be remediated. Colleges of education should be held responsible for inadequate preparation of their graduates, and principals should be blamed for continued incompetence of their teachers. Superintendents should be held accountable for poor performance of principals. You may disagree but I feel that part of the problem with poor performance by educators has been due to the teachers’ unions in many states having too much power and their failure to do any self-policing.
When I was a student (I graduated from high school in 1965), there were no school grades posted on a flashing marquee in front of the school and no publicity about which teachers were doing well. Most parents and students KNEW which teachers were the best and the worst. When the “achievement test” scores came in, parents were invited to the school for a private discussion of their child’s progress or lack thereof. Summer school, remediation, retention, and promotion were also suggested if needed. I never felt afraid when we had “achievement tests”and never was informed of my IQ. Occasionally a gifted student skipped a grade, and it was not unusual for a weak student to be retained. Thinking back, I remember some students who failed a grade yet went on to college—one became an MD and another taught one of my family members economics at the local jr. college.
We “rightwingers” want the basics taught thoroughly in the first 3 grades so that students are fully prepared in reading and arithmetic. Otherwise they will not have the core knowledge needed to grow into “critical thinkers.” We do not want a narrow curriculum and most of us prefer students have an appreciation for excellent art and music and we believe students need play time especially in the early years. We want the content of the curriculum to be uplifting and academically challenging.
Words have evolved. Today I believe most conservatives want what used to be called a “liberal arts” education” with options in high school for career tech courses.
librera Ravitch’s blog Sent: Friday, September 14, 2012 7:18 AM To: bpeters@centurytel.net Subject: [New post] Why Do Conservatives Love High-Stakes Testing?
dianerav posted: “Marcus Winters recommends using value-added assessment to get rid of “ineffective” teachers. His paper was published by the conservative Manhattan Institute, which regularly issues his and others’ critiques of unions, tenure, seniority and any kind of job”
One reformer who had a strong influence in the crafting the high stakes testing and consequences, Sandy Kress, Esq. loves high stakes testing. It would be very easy to like high stakes testing when you are earning tons of money from such clients as Pearson, ETS and the like. What’s not to like about making millions!
I feel that my strength as a teacher is in building foundational skills. Apparently my principal feels this as well as I tend to get the developmentally young, borderline students in my class. I like it this way. In light of VAM I may need to rethink this.
Those who love the bell curve view of the world love high stakes testing. They also espouse tracking and ability grouping and test-in schools. Bloomberg is one of those people. They believe in an elite that will rise and who should be rewarded and lead. Tests in their minds are “objective” therefore they justify their elite conclusions. That is why they are so enamored with Ivy Leaguers in TFA. they themselves always identify with the elite.
Duncan, in my opinion, does what he is told. If he was told that testing is bad for kids, that is what he would say.
Who do you think controls Arne? Certainly not Obama? Really, I wonder who he is listening to….one or two particular people or groups?
Jon Schnaur and the other designers of RTTT… Many work for him. Obama worked with DFERS when he was a senator. Whitney Tilson had a big role in the first campaign. they gave a lot of money and so they got influence.
Wow…so discouraging. I won’t vote for Romney and I just can’t vote for Obama again. He used us and then abandoned us. It won’t matter in my state anyway. It looks like it will come down to a few states to decide the election. I think the opt out movement is one force to pursue…they got nothing without the high stakes tests..talking it up everywhere I can.
It is the rock on which all of this is built
[…] Why Do Conservatives Love High-Stakes Testing? (dianeravitch.net) […]