Archives for category: Testing

Somehow, Andrea Gabor got a copy of most of the New York State English Language Arts Common Core-Aligned State tests.

 

She describes them here.

 

She writes:

 

Once again I am in possession of a bit of educational contraband.

For the second year in a row, I have received a copy of the New York State English Language Arts tests for grades 6 to 8, which were administered in April. (Though, this year, my set appeared incomplete as it contained only books one and two for each grade–not the three books that were included last year and that I was told were given this year. So my analysis here is confined to only two booklets for each grade.)

Anyone who has followed the controversy around the introduction of the New York State’s “common-core aligned” tests, knows that there has been a growing backlash–and not necessarily against the common core itself. Rather, a great many educators object to the quality and the quantity of tests–in addition to six days of “common core” testing, New York kids are now finishing the Measurements of Student Learning (MOSL) tests, the sole purpose of which is to evaluate teachers, as well as field tests for next year’s “common core” tests. In the fall, students as young as kindergarteners endured base-line testing for the MOSL.

Most importantly, educators are outraged by the secrecy in which the tests are cloaked.

 

Pearson, which has a $32 million contract with New York, will not permit teachers or anyone else to see the exams.

 

They are hidden by a gag order.

 

This is insane.

 

The value of tests is to learn what students do and do not know or understand

 

If the students, parents, and teachers are not allowed to review the tests, then nothing can be learned from them.

 

There is no point in having tests that are hidden from the view of those who most need whatever information they provide.

 

Of course, the gag order also protects Pearson from public scrutiny and possible discovery of poorly written or inaccurate questions, like the Pineapple questions.

 

So who benefits from the gag order? Not the students.

Mark Henry, superintendent of the Cypress-Fairbanks district in Texas, stood up and spoke out for common sense and education ethics. In this article, he explains why his district–the third largest in Texas–will not participate in a pilot test to evaluate teachers by student test scores.

He writes:

“This latest movement to “teacher-proof” education places additional fear, anxiety and pressure on professionals who are stressed enough already. I have seen this first-hand with principals and teachers who fret over the STAAR test, a once-per-year high-stakes assessment that measures how a child performed on one test on one day. Is that really learning? I don’t think so. Testing is a key diagnostic tool, and results should be used to assess the progress of students so plans can be developed to address the gaps and deficiencies of each student.
Learning is not a business; it’s a process. Use of a teacher evaluation system tied to standardized test scores alienates educators by trying to transform classrooms into cubicles. There are many more elements that go into teaching and learning than a high-stakes, pressurized test. Tying student test scores to a teacher’s evaluation may improve test scores, but does it improve a child’s educational outcome?”

Henry says there are three reasons that schools fail: mismanagement by school boards and superintendents; ineffective principals; lack of community support.

He does not blame teachers for poor leadership or systemic failure.

He writes:

“Let’s quit trying to “teacher-proof” education and stop the overreliance on data from one high-stakes test. The answers for improvement are recruiting, training and supporting our teaching professionals. Attention to these will deepen the effectiveness of what we do in the classroom and the biggest winners will be our children. ”

Mark Henry is a hero of public education for his willingness to stand against a misinformed and harmful status quo.

This morning I posted a statement by a group of professors at City University of New York in support of the edPTA, which assesses the performance of those who seek certification to enter teaching.

 

Let me make clear that I am not supporting or endorsing either side of this debate but am watching carefully, as I tend to be suspicious of all high-stakes testing.

 

Soon after the post appeared this morning, I received an email from a CUNY professor pointing out that the professors’ union–the Professional Staff Congress– at CUNY opposes edTPA and that those who signed the earlier statement are a minority of the faculty.

 

Due to the opposition of PSC, UUP (United University Professors of State University of New York), and NYSUT (the New York State Union of Teachers), implementation of edTPA has been delayed until June 2015.

 

PSC said this on its website:

 

The Teachers Performance Assessment (edTPA), is a high-stakes assessment for student teachers that includes filmed classroom observations. It has been opposed by PSC, UUP and NYSUT. (NYSUT edTPA resolution.) The State Education Department rushed to implement the controversial teacher certification exam, which was set to be a requirement for teacher certification after May 1, 2015. But education faculty, teachers and their unions pushed back and the implementation of the assessment has been pushed back until June 2015.

In Support of a Performance Assessment of Teaching
June, 2014

Beverly Falk, Professor and Director, Graduate Program in Early Childhood Education, The ​City College of New York

Jeanne Angus, Assistant Professor and Program Director, Graduate Program in Special Education, Brooklyn College

Greg Borman, Lecturer, Secondary Science Education, The City College of New York

Nancy Cardwell, Assistant Professor, Graduate Program in Early Childhood Education, The City ​College of New York

Joni Kolman, Assistant Professor, Department of Teaching, Learning, and Culture, The City College of New York

Geraldine Faria, Assistant Dean, School of Education, Brooklyn College

Christy Folsom, Associate Professor, Childhood Education, Lehman College

Nancy Martin, Adjunct instructor, Childhood Science Education, Brooklyn College

Andrew Ratner, Assistant Professor, Secondary English Language Arts Education, The City College of New York

Deborah Shanley, Professor, Special Education/English, Secondary Education and Dean, School of Education, Brooklyn College

Jacqueline D. Shannon, Associate Professor and Chair
, Department of Early Childhood Education/Art Education
, Brooklyn College

Beverly Smith, Associate Professor, Secondary Mathematics Education, The City College of New York

Christina Taharally, Associate Professor and Director, Graduate Programs in Early Childhood Education, Hunter College

The media and the blogosphere have been filled as of late with discussions about teacher education. Think tanks, states, and the federal government have questioned the efficacy of teacher preparation programs and are proposing accountability measures for them that resemble the high stakes testing in p-12 schools. Many have responded to these problematic policies, which emphasize targets and sanctions rather than supports to improve, with critiques of the dysfunctional consequences they generate: an over-emphasis on tests that narrow the curriculum and the use of value-added measures (such as students’ test scores to evaluate teachers and graduates of teacher education programs) that do not account for all of the complex factors influencing learning. Critics rightly point to these policies as creating disincentives to teach students who traditionally do not score well on tests (those who are poor, new immigrants, who are English language learners, or who have special needs).

Ironically, however, some who are reacting to these negative effects of the test and punish approach are including in their attack an initiative specifically designed to push back against it. They target a performance assessment for teachers designed by the profession for the profession – the edTPA – which calls on prospective teachers to demonstrate through performance (not multiple choice tests) that they have professionally-agreed upon skills and knowledge to enter a classroom ready to teach. The opposers of edTPA make inaccurate claims about it – that it is tied to a high-stakes testing regime and outsourcing evaluations to a private corporation – Pearson; that it demands a single approach to teaching and teacher education; that it usurps academic freedom and faculty control of curriculum; and that it has no research base to evaluate good teaching. Alan Singer’s recent blog post, an example of this opposition, also
claims that edTPA “distracts student teachers from the learning they must do on how to connect ideas to young people and undermines their preparation as teachers.”

We, teacher educators who have used the edTPA, write here to offer a different perspective – to share how it has supported our teaching, our program development, and our students’ learning.

Who we are:

We are teacher educators from the City University of New York, a university comprised of many campuses across NYC that serve a socioeconomically, culturally, racially, and linguistically diverse population of students. We are advocates for equity and access in education. We support culturally-responsive teaching and assessment practices that focus on deep understanding, critical thinking, and analysis of complex issues. We believe assessment should examine what learners know and can do in authentic contexts and that assessment results should be used to support and improve, not target and sanction. Additionally, we support national efforts to make educator preparation more clinically based so that graduates of educator preparation programs are supported in the context of real-life teaching to combine theoretical with practical knowledge so that they can enter their classrooms ready for the incredibly difficult realities of teaching.

Because of these values we welcome the teacher performance assessment (edTPA) –
a performance assessment of teaching developed by hundreds of teachers and teacher educators across the country, in a process led by Stanford University’s Center for Assessment, Learning, and Equity (SCALE), with support from the American Association of Colleges for Teacher Education (AACTE). Based on 25 years of research and practice, this assessment is currently being used in over 500 institutions in 34 states across the United States. In New York State, it is newly required for certification. As result of our first years of experience using it, we find the edTPA to be a tool for our improvement and a valuable guide to effective teaching.

edTPA: A performance assessment of commonly-agreed upon foundational skills
For those who are not familiar with edTPA, here is a brief outline of what it requires. It is a performance assessment that consists of 3 tasks that call on prospective teachers to demonstrate and explain their ability to carry out universally-accepted essentials of good teaching:

• 1) plan 3-5 inter-related learning experiences, taking into consideration the cultural, linguistic, and learning backgrounds of students;

• 2) teach what they have planned, demonstrating through video a segment of a learning experience that is accompanied by a reflective commentary; and

• 3) assess students’ work – examining artifacts of students’ work, including students with learning and language differences needs, for the purpose of using what has been learned from the students’ work to inform future teaching.

Alan Singer and other opponents of edTPA claim that the developers of edTPA (which he inaccurately refers to as including Pearson and New York State) “are trying to sell the public that you learn to teach, not by teaching, but by writing about it. They also want you to believe that they have perfected a magical algorithm that allows them to quickly, easily, and cheaply assess the writing package and accompanying video and instantly determine who is qualified to teach our children.”

Contrary to these claims, we do not see the edTPA as simply a tedious writing exercise.
Indeed the requirements of the edTPA are teaching: planning a curriculum, teaching it for several days, adjusting the plans based on what students are learning, assigning and evaluating student work to shape future teaching is, in fact, what teachers do. In addition to actually teaching, we have experienced edTPA as a useful opportunity to reflect on teaching strategies and students’ needs. We believe, as do the many representatives of professional associations and experienced educators who participated in the development of edTPA, that an essential part of teaching professionalism is being able to explain what we do and why we do it. Only educators who can articulate and defend their practices can uphold the professionalism needed to strengthen our field. Furthermore, we do not agree with the claim that the edTPA demands only one way to demonstrate what is good teaching. The lessons candidates plan are developed by them; the materials they use are chosen by them; the strategies they employ are their choice. The assessment offers a frame that has room for many different approaches. We do not think it takes the artistry out of teaching but instead, by sharpening the focus of our preparation on commonly-agreed upon foundational skills, it enables not only the artistry but also the joy of teaching to take place.

edTPA is controlled and scored by the profession, not the Pearson corporation
Contrary to the claims of Alan Singer and others, the edTPA is not a Pearson assessment. The facts are that Pearson is an operational partner (much like the publisher of a text), responsible for creating and managing the online platform that collects portfolios and delivers them to the teachers and teacher educators who score them. This capacity enables the assessment to be used on a national basis.

Neither is edTPA scored by Pearson. Singer inaccurately claims that “student teachers are not being evaluated by trained field supervisors or cooperating teachers, but by temporary evaluators of questionable qualifications ….who are hired by Pearson.” This is not true. The facts are that edTPA is scored by experienced teachers and teacher educators who are rigorously selected through a process developed by the national consortium of educators who designed edTPA and then taught to examine prospective teachers’ work in relation to commonly-agreed-upon descriptors of exemplary practice (also a process designed by the edTPA consortium).

In our experience, using the edTPA has not taken the development and evaluation of our students out of the hands of our faculty, trained field supervisors, or cooperating teachers. Rather, it has been a helpful complement to our coursework and to the feedback we offer in clinical experiences. In fact, as a result of working with edTPA, not only have we been prompted to develop more coherence in our courses and across our programs, but we have also found that edTPA has prompted our prospective teachers to demonstrate a more intentional and reflective approach to their teaching. Overall, we believe that the edTPA is helping us to better prepare our graduates.

Prospective teachers’ perspectives

The perspectives of prospective teachers who have taken edTPA confirm what we have experienced. In a recent state pilot of edTPA, for example, researchers found that 96% of teacher candidates reported that the edTPA was a positive influence on their learning, pointing especially to how it made them more self-aware and focused them on student learning. More than 90% of teacher educators reported the experience of supporting the edTPA enabled them to reflect on and improve their program design and instruction. For more evidence about the effects of this assessment on candidate learning and teacher education improvement, see http://edtpa.aacte.org/resources.

These research findings are reflected in the remarks of Peter Turner, a graduate of The City College of New York’s School of Education, who noted in a recent interview:

[Although I had student taught already], for me [edTPA] was the first time that I considered every aspect of what it means to be an effective teacher…. The edTPA is a good test because it scaffolds every aspect of what a teacher needs to do. It was a wonderfully educative experience for me.
— Peter Turner, City College of New York

Lehman College graduate, Roshawna Cooper, adds:

Looking at my lesson plans when I was doing edTPA and looking at my older lesson plans when I did methods courses without edTPA, I missed a whole chunk. There was a lot missing. I am so much more mindful of my students now when I am teaching and the effectiveness of my different lessons. I am more mindful about how to build on each lesson to support my students’ skills. And that is really big. ( See http://edtpa.aacte.org/resources/candidate-to-candidate-reflections-on-taking-edtpa)

Safe to practice/ready to teach: Accountability by the profession for the profession
All professions have external certification exams and a commonly-agreed upon set of foundational knowledge. In fact, professions that are responsible for the safety and well-being of humans all require that their certification processes demonstrate not only that new entrants to the profession have knowledge and skills but that they know how to apply these knowledge and skills and are safe to practice with those entrusted to their care. We believe that the edTPA, by asking new teachers to demonstrate that they know how to teach before they are given the privilege of taking responsibility for children’s lives, is a genuine and valid measure of our work. Because edTPA’s rich descriptions and analyses of teaching are aligned with critical commonly-agreed upon elements of effective practice while allowing for individuality and flexibility in content and style, we believe it serves as a useful tool and guide for teaching and is a positive step forward for us as a profession.

Although there is no such thing as a perfect assessment, especially for something as complex as teaching, we believe that edTPA vastly improves the process by which teachers are certified in New York State. It is a mechanism for us as teacher educators to demonstrate the outcomes of our work and to hold ourselves, as a profession, accountable for what we do. It stands as a viable genuine accountability measure for graduates of teacher education as opposed to sole reliance on standardized tests. While its implementation has posed challenges and calls out for changes in “business as usual” in teacher education, we believe these changes are well worth the while because edTPA reflects our aspirations, celebrating what it means to be a teacher and putting into practice the educative aspect of what high quality assessment should be.

Carol Burris has been one of the leading voices in opposition to corporate education reform in New York state. Whenever anyone tries to imply that opposition to the Common Core comes only from the Tea Party, there is Carol Burris–a progressive high school principal–as a counter-example.

 

Burris has led the principals’ revolt against high-stakes testing and against evaluating educators according to student test scores.

 

In this article, she describes the progressive revulsion to Governor Andrew Cuomo, who is more attuned to the interests of major corporations and big-money donors than to parents and educators.

 

Burris sees Fordham law professor Zephyr Teachout as the progressive who is likeliest to challenge Governor Cuomo in a Democratic primary, running against him on his left flank, where he has big vulnerabilities.

 

Given the upset defeat of Eric Cantor in Virginia, no politician can rest easy as they approach an election. In many states and districts, voters are angry and feel cheated.

 

Cantor was surprised. Let’s see if Cuomo coasts to victory, as he expects, and as Cantor expected.

Dr. Iris Rotberg of George Washington University writes that international tests have been fraught with methodological problems for fifty years. None of the problems have been addressed or corrected, yet today the international tests such as PISA are driving educational policy in dozens of nations, all competing for higher test scores.

Rotberg writes:

“The methodological critiques of international test-score comparisons began shortly after the comparisons were first administered 50 years ago, and they have continued. Methodological critiques of research are not unusual, but this situation is quite extraordinary for several reasons. First, the critiques of the international test-score comparisons are extensive and address virtually every aspect of the studies—sampling, measurement, and interpretation. Second, the studies continue to be administered, with few of the critiques addressed, but with continued participation of a large number of countries and other jurisdictions. These massive data collection efforts have been conducted 13 times in the past 18 years. The results of the most recent study, the 2012 Program for International Student Assessment (PISA), were released in December 2013, only a year after the release of the other two major comparisons, the Trends in International Mathematics and Science Study (TIMSS) and the Progress in International Reading Literacy Study (PIRLS) (National Center for Education Statistics, n.d.-a, n.d.-b, n.d.-c). Third, despite the critiques, the studies have had a large impact on political rhetoric, public opinion, and public policies in countries throughout the world. This commentary focuses on PISA, the most recent international comparison released. Although the three international comparisons differ in some respects, the basic methodological problems described here are inherent in international test-score comparisons more generally.”

She adds:

“The international test-score rankings are almost universally interpreted by countries as an indication of the quality of their schools, despite the extensive methodological problems that make it virtually impossible to draw causal relationships between test scores and school quality. We are taking tenuous results and applying them in a questionable way. Even if the rankings were sound, a causal leap from test-score rankings to school quality would be unwarranted given the wide range of other factors that influence the rankings, such as the differences among countries in poverty rates, income distribution, immigration rates, social support services, and the extent to which children participate in academic programs and cram courses outside of school. And beyond all of these variables, there remains the basic question of whether a test score is a fair representation of the complexity and quality of a country’s entire education system. It has proven to be virtually impossible to unravel the cumulative effects of all the uncontrolled variables and then make valid interpretations of the implications of the test-score rankings.”

The international horse race, she says, has led to policies of dubious merit.

And she concludes:

“PISA’s own findings support a transition to studies of individual countries. They show that the proportion of variance in student achievement accounted for by socioeconomic status and other differences within member countries in the Organization for Economic Cooperation and Development (OECD) is nine times greater than the proportion accounted for by differences among OECD countries (OECD, 2010)—a finding that has been obscured by the emphasis on test-score rankings and largely ignored in the public dialogue. It is consistent with a research approach that focuses on problem areas within countries rather than on test-score competitions among countries. It also offers an opportunity to take Einstein’s advice and focus on issues that count, and count only what can be counted. After 50 years of test-score rankings, it’s worth a try.”

Now, as we all know, Bill Gates paid over $2 billion for the Common Core standards. They are supposed to be the linchpin of a coordinated system: standards, tests, teacher evaluations based on test scores, school closings, turnarounds, etc. but a funny thing happened on the way to the millennium. Parents and educators got angry. Some hated the tests. Some hated the standards. Some hated the federal takeover of their public schools. A few states said they would drop the standards.

The Gates Foundation decided the best way to calm the protests was to slow down the implementation. Here is the story in the New York Times.

This afternoon, on one of my rare outings while I recuperate from surgery, I was sitting in the car outside the fish market, when I got an email from reporter Motoko Rich of the Times. She asked what I thought of the moratorium. This is the last quote in the story:

“Some critics of the standards and testing said that a moratorium was not enough.

“If the sanctions and punishments tied to test scores are wrong now — promoting teaching to the test, narrowing the curriculum, cheating, and gaming the system — the sanctions and punishments will still be wrong two years from now,” Diane Ravitch, an education historian and critic of standardized testing in schools, wrote in an email. “The opposition to high-stakes testing will not go away.”

Realcleareducation.com reports that the Gates Foundation favors a moratorium on the consequences of Common Core testing. Since the standards were bought and paid for by the Gates Foundation, it is only right that it should call the shots. Now we know who is in charge of American education. Perhaps the foundation hopes that a delay will defuse the growing movement against Common Core.

Realcleareducation writes:

“Good morning, it’s Tuesday June 10. This morning at RealClearEducation we have news, commentary, analysis, and reports from the education world. This morning Vicki Phillips, Director of Education, College Ready at the Gates Foundation, will call for delaying the attachment of any consequences to the new Common Core State Standards, bolstering the position of those calling for an accountability moratorium. Depending on your perspective that will help or hinder implementation of the new standards more than 40 states are adopting.”

Jeffrey Weiss and Daniel Lathrop report that test results on the state’s STAAR tests have been flat. In addition, they report in the Dallas Morning-News that the STAAR test results show widening gaps between the lowest performing students and their peers.

The great puzzle is why state officials expect scores to go up every year, especially after sustained and large budget cuts. The legislature cut over $5 billion from public schools in 2011. Why do they expect scores to rise with larger class size and other loss of programs and personnel?

“If STAAR results are any measure, Texas is failing its lowest-performing students. Despite government-mandated programs for many thousands of test-challenged kids, three years of scores show no benefit.

In every test at every grade, groups of students who scored lower when STAAR rolled out three years ago are still behind — and in most cases, the gaps are growing.

The problem, said Texas Education Commissioner Michael Williams, is that teaching has not changed to meet the increased difficulty of the tests.

“This is about changing the quality, enhancing the quality of instruction,” he said last week. “I am not suggesting that teachers are not caring or working hard. I am suggesting that what we are requiring of teachers today is much more difficult than what we required in the past.”

Last week, Williams announced that he would be granting new flexibility for school districts to use $1.3 billion a year in federal money for low-income students. He said that the changes were to help close student achievement gaps. And that the state would offer examples of successful programs.

The examples have not yet been identified.

Others say the gap problem is primarily a lack of resources — low school funding. Or unrealistic expectations — the new standards are too high. Or a problem with the tests — they’re not a good measure of what students learn.”

State Commissioner Williams is not an educator. In his last position, he “regulated” the energy industry, which in Texas is very lightly regulated. Why he thinks he has the knowledge to tell teachers how to teach is one of the signs of our dysfunctional age.

Why Texas–and the nation–expects test scores to go up every year is a mystery.

Why Texas–and the nation–expect low performing kids to have higher test scores as their schools’ budget is cut is another mystery.

Why politicians think the way to raise test scores is to make the tests harder is baffling.

Sue Legg used to be in charge of assessment for the state of Floridea, before the current reign of educational inanity took hold. She writes:

 

QUANTITY VERSUS QUALITY IN TESTING

Back in the late 90s, psychometricians were searching for ways to counter attacks against multiple choice testing. Opponents argued that essay exams were needed to measure higher order thinking skills. Too often multiple choice tests relied on fact based information or esoteric vocabulary as indirect measures of analytical reasoning. In response, we added essay components to state-wide assessments. Essay scoring, however, had its own problems. Readers are people–they have their own biases about essay scoring. I remember Mark Reckase, then at ACT, saying that it would take the average score of 7 readers to achieve the reliability of a multiple choice test. It was a conundrum. Do we trade the validity of the essay that could measure critical thinking for the reliability of a fact-based multiple choice test?

Instead, we looked for ways to combine and improve the two types of questions. At the same time, we worked to capitalize on the possibilities for using computers to improve testing and instruction. I remember the excitement over the possibilities of using computers to adapt multiple choice tests to better measure students’ abilities without spending endless hours in testing laboratories. Essay scoring could also be more efficient if computers could substitute for some, but not all human readers. Testing could become part of the learning process as well as less onerous.

Enter the law of unintended consequences. Instead of helping diagnose student learning problems, testing became the face of an accountability driven political agenda to reform rather than fund educational change. Even though testing has now become extremely expensive, it costs less than attracting and retaining high quality teachers and providing the support struggling students need to learn.

We do not have much to show for all the emphasis on accountability driven reform. Serous efforts to evaluate progress have come up short. Excessive testing does not make better teachers, better schools, or better students. Instead, legislators have created a huge profit driven testing industry, but even that industry cannot offer the advantages of new, more efficient ways to measure student learning for everyone. States are not willing to limit the frequency of testing to make it meaningful and affordable. Even the infrastructure needed to deliver these online, adaptive exams is limited.

Equally bad, the temptation to recreate the old paper and pencil drill and practice worksheets on the computer and call it innovative instruction is proving hard for online learning companies and legislators to resist. They are easy to develop and profitable. The question is how we can force reform minded legislators to recognize that there is a difference between quality and quantity. Online learning and testing has possibilities to improve teaching and learning that are beyond exciting. Unfortunately, they may be beyond the critical thinking and problem solving skills of the current wave of school reformers. One wonders if legislators are up to this test?