This is a must-read article.
One of the best education writers in New York State is Gary Stern of lohud.com, which covers the Lower Hudson region. This article shows how the passing marks (“cut scores”) were set for the state’s Common Core tests. It is a story that should have appeared in the New York Times. The State Education Department likes to boast that the cut scores are set by teachers. This is supposed to make them legitimate, on the assumption that the teachers have reasonable expectations and know the students’ capacity. All 95 teachers who participated in the process of setting cut scores were required to sign a confidentiality agreement, but Gary Stern persisted and found 18 who were willing to talk about the process without violating the agreement.
What Gary Stern found was that Pearson called the shots, not the teachers.
Here are some quotes.
“How does the state determine the crucial break between a 2, which means that a student is not quite proficient in, say, fifth-grade math, and a 3, which signifies that he or she is on track for college?
“These scoring scales were set last summer by a group of 95 educators that the state gathered at a hotel in Troy for several days. Teachers, administrators and college professors from across New York signed confidentiality agreements and were given the task of setting the cuts between 1 and 2, 2 and 3, and 3 and 4 for the new tests. But the scores would be widely questioned and even ridiculed after one-third of New York students were deemed to be on pace……”
“To most parents, passing a test means earning 65 out of 100 points. Cut and dried.
“The process of setting a scoring “scale” and cut scores for an annual test, based on all-important, predetermined goals, is an entirely different animal that is not easily described. In fact, the panelists met to set the 1-4 cut scores after students took the first new tests in spring 2013 and the raw data was in.
“It’s like you’re jumping over a hurdle that’s 2 feet high, but after you jump they say it was 3 feet and you missed,” said Cary Grimm, another panelist who is math chairman for the Longwood school district on Long Island.
“In brief, panelists were assigned to small groups that looked at several grades’ exams in math or English language arts. They were given detailed descriptions of what students should know in each grade — prepared by state officials and experts from Pearson Inc., the mega-corporation signed to create New York’s tests…..”
“Panelists were told whether various cut scores would jibe with research on what it supposedly takes to succeed in college.
“Jane Arnold, an English professor at SUNY Adirondack, said the Pearson people provided confusing data that didn’t seem to apply to grades 3-5, her group’s focus.
“Then they gave us a chance to change our minds,” she wrote in a statement. “In other words, we all knew that most of the student scores would be substandard…..”
“Maria Baldassarre Hopkins, assistant professor of education at Nazareth College in Rochester, said the process was driven by the introduction of outside research about student success.
“I question how much flexibility and freedom the committee really had,” she said. “The process was based solely on empirical data, on numbers. … There are ways to make the numbers do what you want them to do.”
“Tina Good, coordinator of the Writing Center at Suffolk County Community College, said her group produced the best possible cut scores for ELA tests in grades 3 to 6 — playing by the rules they were given.
“We worked within the paradigm Pearson gave us,” she said. “It’s not like we could go, ‘This is what we think third-graders should know,’ or, ‘This will completely stress out our third-graders.’ Many of us had concerns about the pedagogy behind all of this, but we did reach a consensus about the cut scores.”
“Eva Demyen, superintendent of the Deer Park district on Long Island, said she still doesn’t grasp how the state determined that two-thirds of students were not proficient in English and math.
“How they got the 33 percent (passing) was beyond us,” she wrote. “It just seemed very strange to me … and I’m a mathematician!….”
“Another panelist, Karen DeMoss, a professor of education at Wagner College on Staten Island, said she is increasingly convinced that standardized testing is “scarring” students and not promoting achievement.
“Our process was perfectly fine, and the Common Core standards may be the best thing the country has ever had in education,” DeMoss said. “The problem is the underlying assumption that these tests are helping us. They’re not. Pearson’s tests were unbelievably bad, the worst I’ve seen, and the reality of using tests designed to rank students is something we haven’t gotten our heads around.”
There are at least three lessons are to be learned from this fiasco: one, it was Pearson, not the educators, that decided what students should know; two, Pearson’s standards will cause massive failure wherever they are used; three, as many panelists noted, teachers did not have the training to teach the standards.
And there is one more lesson: if the standards themselves are developmentally inappropriate–if the tests expect fifth-graders to learn material that is appropriate for seventh graders, failure is inevitable. Unless, that is, Pearson and the State Education Department decide to lower the cut scores to give the illusion of progress.
As Gary Stern wrote: “A 2006 primer on cut scores prepared by the Educational Testing Service found that cut scores can be reliable, but are based on a group’s opinions.
“It is impossible to prove that a cut score is correct,” the report said.
Remember that the cut score is NOT an objective measure. It is a judgment call, a matter of group opinion, shaped by assumptions, and it can be manipulated to make scores appear higher or lower, depending on what the state wants. If New York’s scores go up, it means that the State Education Department decided to reduce parent anger by lowering the failure rate.
This is what happened in New York. It is wrong, it is cynical, it is misguided. Thousands of children were falsely labeled as failures. This is not good education. This is not about the needs of children. This is institutional incompetence.
If your state plans to use Pearson and PARCC for Common Core testing, consider this a cautionary tale. As Peter Greene writes in his blog,
“In fact, among the CCSS supporters who spoke (and really– did you think NYS would fill this committee with people who didn’t love the Core), there was a recognition that the implementation is a hash and the tests are a bogus joke. Yes, they haven’t figured out that what we’ve got is exactly what the Core were designed to give us, but at least they recognize some of the suckage, and not simply from a practical political calculus angle (and remember– everyone must take calculus now). This is undoubtedly part of the reason that CCSS enjoys the kind of support in NYS usually reserved for politicians who cannot keep their private parts off the internet.
“It’s an illuminating batch of reportage, well worth your time to read. Because you may not live in New York, but wherever you are in America, you’re still living in the United States of Pearson.”
