In Support of a Performance Assessment of Teaching
June, 2014

Beverly Falk, Professor and Director, Graduate Program in Early Childhood Education, The ​City College of New York

Jeanne Angus, Assistant Professor and Program Director, Graduate Program in Special Education, Brooklyn College

Greg Borman, Lecturer, Secondary Science Education, The City College of New York

Nancy Cardwell, Assistant Professor, Graduate Program in Early Childhood Education, The City ​College of New York

Joni Kolman, Assistant Professor, Department of Teaching, Learning, and Culture, The City College of New York

Geraldine Faria, Assistant Dean, School of Education, Brooklyn College

Christy Folsom, Associate Professor, Childhood Education, Lehman College

Nancy Martin, Adjunct instructor, Childhood Science Education, Brooklyn College

Andrew Ratner, Assistant Professor, Secondary English Language Arts Education, The City College of New York

Deborah Shanley, Professor, Special Education/English, Secondary Education and Dean, School of Education, Brooklyn College

Jacqueline D. Shannon, Associate Professor and Chair
, Department of Early Childhood Education/Art Education
, Brooklyn College

Beverly Smith, Associate Professor, Secondary Mathematics Education, The City College of New York

Christina Taharally, Associate Professor and Director, Graduate Programs in Early Childhood Education, Hunter College

The media and the blogosphere have been filled as of late with discussions about teacher education. Think tanks, states, and the federal government have questioned the efficacy of teacher preparation programs and are proposing accountability measures for them that resemble the high stakes testing in p-12 schools. Many have responded to these problematic policies, which emphasize targets and sanctions rather than supports to improve, with critiques of the dysfunctional consequences they generate: an over-emphasis on tests that narrow the curriculum and the use of value-added measures (such as students’ test scores to evaluate teachers and graduates of teacher education programs) that do not account for all of the complex factors influencing learning. Critics rightly point to these policies as creating disincentives to teach students who traditionally do not score well on tests (those who are poor, new immigrants, who are English language learners, or who have special needs).

Ironically, however, some who are reacting to these negative effects of the test and punish approach are including in their attack an initiative specifically designed to push back against it. They target a performance assessment for teachers designed by the profession for the profession – the edTPA – which calls on prospective teachers to demonstrate through performance (not multiple choice tests) that they have professionally-agreed upon skills and knowledge to enter a classroom ready to teach. The opposers of edTPA make inaccurate claims about it – that it is tied to a high-stakes testing regime and outsourcing evaluations to a private corporation – Pearson; that it demands a single approach to teaching and teacher education; that it usurps academic freedom and faculty control of curriculum; and that it has no research base to evaluate good teaching. Alan Singer’s recent blog post, an example of this opposition, also
claims that edTPA “distracts student teachers from the learning they must do on how to connect ideas to young people and undermines their preparation as teachers.”

We, teacher educators who have used the edTPA, write here to offer a different perspective – to share how it has supported our teaching, our program development, and our students’ learning.

Who we are:

We are teacher educators from the City University of New York, a university comprised of many campuses across NYC that serve a socioeconomically, culturally, racially, and linguistically diverse population of students. We are advocates for equity and access in education. We support culturally-responsive teaching and assessment practices that focus on deep understanding, critical thinking, and analysis of complex issues. We believe assessment should examine what learners know and can do in authentic contexts and that assessment results should be used to support and improve, not target and sanction. Additionally, we support national efforts to make educator preparation more clinically based so that graduates of educator preparation programs are supported in the context of real-life teaching to combine theoretical with practical knowledge so that they can enter their classrooms ready for the incredibly difficult realities of teaching.

Because of these values we welcome the teacher performance assessment (edTPA) –
a performance assessment of teaching developed by hundreds of teachers and teacher educators across the country, in a process led by Stanford University’s Center for Assessment, Learning, and Equity (SCALE), with support from the American Association of Colleges for Teacher Education (AACTE). Based on 25 years of research and practice, this assessment is currently being used in over 500 institutions in 34 states across the United States. In New York State, it is newly required for certification. As result of our first years of experience using it, we find the edTPA to be a tool for our improvement and a valuable guide to effective teaching.

edTPA: A performance assessment of commonly-agreed upon foundational skills
For those who are not familiar with edTPA, here is a brief outline of what it requires. It is a performance assessment that consists of 3 tasks that call on prospective teachers to demonstrate and explain their ability to carry out universally-accepted essentials of good teaching:

• 1) plan 3-5 inter-related learning experiences, taking into consideration the cultural, linguistic, and learning backgrounds of students;

• 2) teach what they have planned, demonstrating through video a segment of a learning experience that is accompanied by a reflective commentary; and

• 3) assess students’ work – examining artifacts of students’ work, including students with learning and language differences needs, for the purpose of using what has been learned from the students’ work to inform future teaching.

Alan Singer and other opponents of edTPA claim that the developers of edTPA (which he inaccurately refers to as including Pearson and New York State) “are trying to sell the public that you learn to teach, not by teaching, but by writing about it. They also want you to believe that they have perfected a magical algorithm that allows them to quickly, easily, and cheaply assess the writing package and accompanying video and instantly determine who is qualified to teach our children.”

Contrary to these claims, we do not see the edTPA as simply a tedious writing exercise.
Indeed the requirements of the edTPA are teaching: planning a curriculum, teaching it for several days, adjusting the plans based on what students are learning, assigning and evaluating student work to shape future teaching is, in fact, what teachers do. In addition to actually teaching, we have experienced edTPA as a useful opportunity to reflect on teaching strategies and students’ needs. We believe, as do the many representatives of professional associations and experienced educators who participated in the development of edTPA, that an essential part of teaching professionalism is being able to explain what we do and why we do it. Only educators who can articulate and defend their practices can uphold the professionalism needed to strengthen our field. Furthermore, we do not agree with the claim that the edTPA demands only one way to demonstrate what is good teaching. The lessons candidates plan are developed by them; the materials they use are chosen by them; the strategies they employ are their choice. The assessment offers a frame that has room for many different approaches. We do not think it takes the artistry out of teaching but instead, by sharpening the focus of our preparation on commonly-agreed upon foundational skills, it enables not only the artistry but also the joy of teaching to take place.

edTPA is controlled and scored by the profession, not the Pearson corporation
Contrary to the claims of Alan Singer and others, the edTPA is not a Pearson assessment. The facts are that Pearson is an operational partner (much like the publisher of a text), responsible for creating and managing the online platform that collects portfolios and delivers them to the teachers and teacher educators who score them. This capacity enables the assessment to be used on a national basis.

Neither is edTPA scored by Pearson. Singer inaccurately claims that “student teachers are not being evaluated by trained field supervisors or cooperating teachers, but by temporary evaluators of questionable qualifications ….who are hired by Pearson.” This is not true. The facts are that edTPA is scored by experienced teachers and teacher educators who are rigorously selected through a process developed by the national consortium of educators who designed edTPA and then taught to examine prospective teachers’ work in relation to commonly-agreed-upon descriptors of exemplary practice (also a process designed by the edTPA consortium).

In our experience, using the edTPA has not taken the development and evaluation of our students out of the hands of our faculty, trained field supervisors, or cooperating teachers. Rather, it has been a helpful complement to our coursework and to the feedback we offer in clinical experiences. In fact, as a result of working with edTPA, not only have we been prompted to develop more coherence in our courses and across our programs, but we have also found that edTPA has prompted our prospective teachers to demonstrate a more intentional and reflective approach to their teaching. Overall, we believe that the edTPA is helping us to better prepare our graduates.

Prospective teachers’ perspectives

The perspectives of prospective teachers who have taken edTPA confirm what we have experienced. In a recent state pilot of edTPA, for example, researchers found that 96% of teacher candidates reported that the edTPA was a positive influence on their learning, pointing especially to how it made them more self-aware and focused them on student learning. More than 90% of teacher educators reported the experience of supporting the edTPA enabled them to reflect on and improve their program design and instruction. For more evidence about the effects of this assessment on candidate learning and teacher education improvement, see http://edtpa.aacte.org/resources.

These research findings are reflected in the remarks of Peter Turner, a graduate of The City College of New York’s School of Education, who noted in a recent interview:

[Although I had student taught already], for me [edTPA] was the first time that I considered every aspect of what it means to be an effective teacher…. The edTPA is a good test because it scaffolds every aspect of what a teacher needs to do. It was a wonderfully educative experience for me.
— Peter Turner, City College of New York

Lehman College graduate, Roshawna Cooper, adds:

Looking at my lesson plans when I was doing edTPA and looking at my older lesson plans when I did methods courses without edTPA, I missed a whole chunk. There was a lot missing. I am so much more mindful of my students now when I am teaching and the effectiveness of my different lessons. I am more mindful about how to build on each lesson to support my students’ skills. And that is really big. ( See http://edtpa.aacte.org/resources/candidate-to-candidate-reflections-on-taking-edtpa)

Safe to practice/ready to teach: Accountability by the profession for the profession
All professions have external certification exams and a commonly-agreed upon set of foundational knowledge. In fact, professions that are responsible for the safety and well-being of humans all require that their certification processes demonstrate not only that new entrants to the profession have knowledge and skills but that they know how to apply these knowledge and skills and are safe to practice with those entrusted to their care. We believe that the edTPA, by asking new teachers to demonstrate that they know how to teach before they are given the privilege of taking responsibility for children’s lives, is a genuine and valid measure of our work. Because edTPA’s rich descriptions and analyses of teaching are aligned with critical commonly-agreed upon elements of effective practice while allowing for individuality and flexibility in content and style, we believe it serves as a useful tool and guide for teaching and is a positive step forward for us as a profession.

Although there is no such thing as a perfect assessment, especially for something as complex as teaching, we believe that edTPA vastly improves the process by which teachers are certified in New York State. It is a mechanism for us as teacher educators to demonstrate the outcomes of our work and to hold ourselves, as a profession, accountable for what we do. It stands as a viable genuine accountability measure for graduates of teacher education as opposed to sole reliance on standardized tests. While its implementation has posed challenges and calls out for changes in “business as usual” in teacher education, we believe these changes are well worth the while because edTPA reflects our aspirations, celebrating what it means to be a teacher and putting into practice the educative aspect of what high quality assessment should be.

On May 3, I received an email from Professor Raj Chetty of Harvard University, informing me that his famous paper on value-added assessment of teachers was being published by the American Economic Review. The paper has three authors: in addition to Chetty, the other authors include John Friedman and Jonah Rockoff, also at Harvard. When the paper was first released, it was reported on the front page of the New York Times, one of the authors discussed it on the PBS Newshour, and President Obama referred to it in his 2012 State of the Union address.

The New York Times story appeared on January 6, 2012. it began thus:

“WASHINGTON — Elementary- and middle-school teachers who help raise their students’ standardized-test scores seem to have a wide-ranging, lasting positive effect on those students’ lives beyond academics, including lower teenage-pregnancy rates and greater college matriculation and adult earnings, according to a new study that tracked 2.5 million students over 20 years.”

The reporter noted that the effect of a single “high-value” teacher was actually quite modest: “The average effect of one teacher on a single student is modest. All else equal, a student with one excellent teacher for one year between fourth and eighth grade would gain $4,600 in lifetime income, compared to a student of similar demographics who has an average teacher. The student with the excellent teacher would also be 0.5 percent more likely to attend college.” But think of the aggregate effect on an entire classroom: “Replacing a poor teacher with an average one would raise a single classroom’s lifetime earnings by about $266,000, the economists estimate. Multiply that by a career’s worth of classrooms.” President Obama cited the aggregate income gain for a classroom in his State of the Union address 18 days later.

This was the takeaway from the authors, as reported in the New York Times:

“The authors argue that school districts should use value-added measures in evaluations, and to remove the lowest performers, despite the disruption and uncertainty involved.

“The message is to fire people sooner rather than later,” Professor Friedman said.

“Professor Chetty acknowledged, “Of course there are going to be mistakes — teachers who get fired who do not deserve to get fired.” But he said that using value-added scores would lead to fewer mistakes, not more.

“Still, translating value-added scores into policy is fraught with problems. Judging teachers by their students’ test scores might encourage cheating, teaching to the test or lobbying to have certain students in class, for instance.”

The Chetty, et al, study supported VAM, which was the central feature of Race to the Top. Fire teachers sooner rather than later. One great teacher can produce lifetime gains.

Over the past few years, as more districts have implemented VAM, it has turned out to be far more complicated than the economists predicted to determine which teachers would produce great scores year after year, and which would not. Teachers were rated effective one year, ineffective the next year. Those who taught English learners, the gifted, and students with disabilities were less likely to get big gains. It turned out that VAM is affected by the composition of the classroom, since students are not randomly assigned.

But their paper continues to be the lodestar of VAM research.

Whereas it had originally appeared as a single paper published by the National Bureau of Economic Research, the editors suggested the paper was so important that it should be split into two papers and published separately. The last time this had happened was in 1971, for papers on taxation that had won two Nobel Prizes.

Here are the papers.

Click to access w19423.pdf

Click to access w19424.pdf

Professor Chetty’s email was addressed to me and Audrey Amrein-Beardsley, who has written extensively and critically about value-added assessment. In addition to her recently published book on VAM—Rethinking Value-Added Models in Education: Critical Perspectives on Tests and Assessment-Based Accountability—she writes a blog called VAMboozled that I often cite.

For the record, I have never met Raj Chetty, and I have met Beardsley once, when she interviewed me for an oral history archive.

I asked Beardsley if she would be willing to review the latest iteration of this now famous study of VAM, and she did, here on her blog.

Beardsley notes that there is a divide between econometricians, like Chetty, Friedman, and Rockoff, and educational researchers, who often feel some obligation to visit classrooms and see the effects of policies, not just analyze data from a great distance, without reference to context or something like reality.

Professor Chetty and I exchanged several emails. I asked for his permission to post our exchange. He said that he preferred that I not post his comments, which were invariably polite, but of course I was free to post my comments to him.

So here goes. This was my first response:

Dear Professor Chetty,

I certainly agree that teachers are valuable. I had some wonderful teachers
as I was growing up, also some mediocre ones, and a few really bad ones. I
went to an ordinary public school system in Houston, not an elite private
school.

I wish that this sentiment about the value of teachers was all that came
from your vast publicity machine.

Instead, we get more high-stakes testing, more test prep, more phony claims
that the work of my fourth grade or fifth-grade teacher was responsible for
my not getting pregnant when I was 15. Maybe my lifetime income was
increased by my sixth-grade teacher, though I doubt it. Funny, I was one of
eight children. We all had the same teachers, and we all turned out
differently. Some of us did well in school, others nearly flunked out. Was
it the fault of our teachers?

I know you love your celebrity–and hobnobbing with Obama and Duncan and
supporting their emphasis on testing and firing teachers sooner rather than
later—but think of the harm that you do to millions of children and their
teachers by the way you publicize your work. Do you feel good every time you
read about a teacher who is graded based on the work of children she never
taught? Or the “highly effective” teacher who was rated ineffective the next
year based on test scores? Or the precipitous decline in the number of
people who want to be teachers because of the non-stop attacks on teachers?
I don’t think your positive message is getting through. All people hear is
that you want those lousy teachers whose kids get low scores to be fired.
Now.

Diane Ravitch

On May 5, I wrote to both Raj and Audrey (we had reached a first-name basis):

Raj and Audrey,

I don’t know whether my thoughts advance or retard this informed discussion.

I look at the Chetty, etc. study as comparable to a pilot in a bomber
dropping a bomb on a city 30,000 feet below. He didn’t construct the bomb,
he doesn’t know how it hurts the people below, he can’t be held responsible
if his good intentions went wrong.

I invite you to read this blog by a teacher in Oklahoma:
http://bluecerealeducation.blogspot.com/2014/05/ms-bullens-data-rich-year.html

The odds are that he never heard of Raj Chetty. But look what Raj Chetty has
done to the quality of education, the students, and the teachers in
Oklahoma. Is this something to be proud of?

Your work–not yours alone, of course–has encouraged a technocratic
approach to education that would never be tolerated in our nation’s elite
private schools.

The pursuit of higher test scores on stupid multiple-choice standardized
tests does not improve education: it corrupts it.

Those who care deeply about humanistic education, about the life of the
mind, about deep learning, find your work–no matter how technically
perfect–utterly appalling. It drains education of joy and discovery and
makes everyone a slave to Pearson.

I would love to discuss this further with you over a glass of wine. I can’t
believe you do not understand the pernicious effects of your famous study,
featured on the first page of the New York Times, on the PBS Newshour, and
in President Obama’s State of the Union Address.

It seems to be my life work to insist that education is far, far more than a
score on a standardized test. Somehow, I suspect you agree. You are far too
intelligent not to.

Diane

Later on the same day, May 5, Raj responded, and I wrote:

Thanks, Raj,

A question and a comment.

My question: Could I publish our exchange on my blog? I get about 25,000-40,000 readers daily. But I would publish nothing without your permission.

My comment: Race to the Top has incentivized the use of VAM in most states. Your study has been cited by Obama and Duncan as evidence that they are on the right track, that it is “bad teachers,” not poverty, that cause low test scores.

Based on the real-world effects of VAM on real children and real teachers, I conclude that VAM has limited use, perhaps informative in looking at the effects of policies and programs (faithfully enacted, which they seldom are) in a school or a district, but of zero value in assessing individual teacher quality. As you must know by now, the ratings for individual teachers are unstable, and may change if a different test is used or unstable for no apparent reason at all. Teachers intuitively know that their ratings reflect the composition of the class, not their “quality” or efficacy as teachers. Even if VAM did work–and it does not–it would keep every teacher singularly focused on standardized tests, which narrow the curriculum, encourage schools and teachers to avoid the neediest students, promote test prep and cheating, and have other perverse effects.

At the end of the day, I as a mother and grandmother would not want my offspring to be enrolled in a school where standardized tests dominate teaching and learning. And that is precisely what VAM is doing to our nation’s public schools.

My third grandson enters third grade in a New York City public school next September. I hope by then that the opt out movement has grown so strong that teachers cannot be subjected to unfair and inaccurate VAMs. I will do whatever I can to encourage parents in every school district in the U.S. to keep their children home on testing day. That seems to be the only way that the giant standardized testing machine can be stopped.

Your work has been crucial in promoting standardized testing as the measure of teacher quality, even though major scholarly organizations disagree (the American Educational Research Association, the National Academy of Education, the American Statistical Association).

If you have modified your views (message: “fire teachers sooner rather than later”); if you have learned anything new since you first introduced your findings, I would love to know about it.

I repeat that I do not have the technical ability to argue algorithms with you. Your study may be technically brilliant. But its consequences for the quality of education and the lives of children and teachers have been disastrous. In its current application, it is Junk Science. Since I feel certain you don’t want to be remembered in history as the economist who sponsored Junk Science and treated children as data points, I hope you will give me reason to believe that you have rethought the conclusions of your study and provided clear warnings about the limitations and misuses of VAM.

Diane

We ended with the understanding that I would not quote his words or paraphrase them. I think I was true to that understanding.

Now, as I told him, I am not an economist, and I lack the technical proficiency to critique his paper. Maybe it will win two or three Nobel prizes. If all it says is that teachers are valuable, I agree. If it says that teacher affect eternity, I agree.

But if he really expects me to believe that my fifth-grade teacher (or was it my fourth-grade teacher) caused me to get higher test scores, and that because of her and my higher test scores, I did not get pregnant when I was 15, I think this is just plain silly.

This strikes me as the kind of study that brings huzzahs from economists for its technical precision, but is unrelated to the messiness of real life. The numbers may all add up, but there are no living, breathing students or teachers here, just data.

It is so incredibly frustrating to me to see economists and policymakers playing with the lives of children and teachers as if they were ants seen from a far distance or merely data points. I recommend to my new friend Raj a book by Yale Professor James C. Scott titled “Seeing Like a State: How Certain Schemes to Improve the Human Condition Have Failed.” It changed my life. Maybe it will change his too.