Archives for category: Testing
Newer Entries »

New York’s Bargain Basement Tests

By dianeravitch
April 28, 2012 //
13

When I read Gail Collins’ nifty column about the pineapple fiasco, I realized why New York ended up using a ridiculous story that was recycled from tests given in several other states. Wherever that pineapple story appeared on the state tests, the kids said it was a stupid story with stupid questions. Pearson didn’t listen and didn’t care. They just sent it on to the next state test. (http://www.nytimes.com/2012/04/28/opinion/collins-a-very-pricey-pineapple.html?_r=1&src=me&ref=general).

New York got the pineapple story because NY is paying Pearson only $32 million for five years of tests. Texas is paying Pearson $500 million for five years of tests. That means that Texas gets the shiny, new questions–the ones that make sense-and NY gets the recycled remainders, the ones that no one else wanted

You get what you pay for.

Diane

Categories New York, Testing

NY State Education Department is a Problem

By dianeravitch
April 25, 2012 //
3

Ten years ago, the New York State Education Department got embroiled in a very embarrassing scandal. A vigilant parent discovered that many passages on the Regents’ exam in English had been heavily rewritten to remove any references to race, religion, gender, sex, and a bunch of other topics. The parent’s discovery merited a front-page story in the New York Times and caused the State Commissioner of Education to promise that it would never happen again. (http://gothamschools.org/2012/04/23/states-promise-to-bar-edited-test-passages-repeats-2002-vow/)

Of course, it did happen again. On the very next administration of the Regents, there was a single stanza from Matthew Arnold’s famous poem, “Dover Beach,” not the full four stanzas of the poem. The last stanza begins, “Ah, love, let us be true to one another!” but the state changed it to, “Ah, friend, let us be true to one another.”

How soon they forget. Now with the Pineapple story, we learn that they never stopped meddling with the text of passages. They can’t help themselves. They think they can write better than Elie Wiesel, better than Isaac Bashevis Singer, better than Daniel Pinkwater.

I told Gotham Schools, trying to be charitable, that maybe they forgot the promise made a decade ago. But I was too kind. The NY SED just isn’t very smart. They send out tests with egregious errors, and the Commissioner blames the teachers who reviewed it. The math tests that will be given tomorrow has errors, but no one at SED will admit they are culpable. Stuff happen is what they say. Don’t blame us.

In 2010, the Regents commissioned a study of the state tests and independent experts said the tests had been dumbed down, the cut scores had been lowered in an effort to meet NCLB’s absurb target. So test scores across the state plummeted to reflect the reality that the SED had gamed the system. But was anyone at SED held accountable? Of course not.

The moral of the story: Accountability begins at the top.

Diane

Categories New York, Testing

I hope you will sign this important resolution

By dianeravitch
April 25, 2012 //
8

This is the National Resolution Against High-Stakes Testing.

Please read it: http://timeoutfromtesting.org/nationalresolution/

If you agree, I hope you will sign it.

Let’s all raise our voices together against this nutty regime of ranking, rating, grading, evaluating every child, adult, school. Anything that moves.

Say no. Make them stop.

Diane

Categories Testing

Comment to a comment: why release test items?

By dianeravitch
April 24, 2012 //
9

A reader said it would be expensive to release all test items as the test publishers would have to spend lots more money creating new ones.

Actually there is another way to think about all this. Release all the items as a public bank of thousands of items. Each year’s tests can be reviewed and howlers would be omitted from future use. No student could possibly take or memorize every item. The bank of tens of thousands of items would make good fodder for test prep.

Let’s face it. The items are recycled now. It is a time-honored practice in NY state for teachers of Regents exams to use old versions of the Regents. Sometimes the old questions appear again–either in exactly the same form or so slightly modified that it doesn’t matter.

Why did Pearson charge NY $32 million for a lot of items that had been pulled out of its old testing bank, recycled from other states. The Pineapple story had been used and ridiculed in other states but Pearson didn’t retire it. Now they have pineapple juice on their corporate face. Why did Pearson charge Florida $250 million for a test that students took in two shifts, with no change in the items? Now Pearson will up their fee to pay themselves for test security.

At some point, the public will see all this as a great farce that takes money away from instruction and plows it into redundant and error-prone testing.

Diane

Categories New York, Pearson, Testing

Pay attention to Peer Assistance & Review to replace high-stakes testing for evaluation

By dianeravitch
April 24, 2012 //
6

Great common on my Edweek blog. Often I wish I had a way to call attention to smart comments like this one. My new blog gives me the chance to give additional attention to the ideas of thoughtful writers. This one comments as “laborlawyer.”

Objecting to high-stakes-testing cannot stop high-stakes-testing unless those objecting can offer a reasonable alternative to high-stakes-testing as a way to identify/remove ineffective teachers. The high-stakes-testing supporters argue that there are ineffective teachers in the schools, that current teacher evaluation systems are not identifying/removing those ineffective teachers, and that these ineffective teachers are a major cause of poorly-performing schools (that is, of low test scores in inner-city schools). These arguments are superficially compelling. It’s true that there are ineffective teachers in the schools (just as there are ineffective employees everywhere) and virtually all citizens have personal recollections of ineffective teachers from their own school days. It’s also true that current teacher evaluation systems are not identifying/removing ineffective teachers — in most school systems, the traditional principal-observes-and-evaluates evaluation system results in very few, if any, discharges. It’s probably not true that ineffective teachers are a major cause of poorly-performing schools (that is, the poorly performing schools are concentrated in low-income areas while the ineffective teachers, even if somewhat more common in the low-income areas, are certainly not concentrated in the low-income areas to the same extent as the low test scores are). But — it’s impossible to prove that ineffective teachers are not a major problem and, in any event, arguing in favor of ineffective teachers has zero appeal. High-stakes-testing is an inexpensive solution to the identify/remove-ineffective-teacher issue. So — to win the debate, opponents of high-stakes-testing must provide an alternative solution. An obvious possibility is a peer-review evaluation system similar to that used in Montgomery County, MD schools since 2001. This system — called “PAR” — has been amazingly successful at removing ineffective teachers with over 500 teachers discharged or resigned-in-lieu-of-PAR-evaluation. The teachers union supports PAR. There have been few litigation challenges to the discharges. The overwhelming majority of teachers think the system is fair. There is no high-stakes-testing, with all its adverse side effects. Briefly, principals identify teachers as possibly poor-performers; senior consulting teachers (who do not report to the principal) intensively monitor/evaluate the identified teachers; a consulting-teachers/principals committee makes the final discharge decision. It’s unclear why PAR has received so little public attention. The NY Times wrote a column praising PAR. But, the media and ed bloggers have otherwise largely ignored it. Opponents of high-stakes-testing should study and publicize PAR — or something like it — as an inexpensive, productive alternative to the destructive high-stakes-testing as a way to identify/remove ineffective teachers.

Categories Teacher Evaluations, Testing

How stupid items get onto standardized tests

By dianeravitch
April 24, 2012 //
23

Many people have wondered how the New York State Education Department permitted the nonsensical story about the pineapple and the hare to get onto the state test.

This is not the first time a really bad reading passage got onto the test and it won’t be the last.

State Commissioner John King was quick to issue a defensive statement saying that people were reading the story “out of context,” as if the full story made sense (it didn’t). And he was quick to pin the blame on teachers, who supposedly had reviewed all the test items. It was the teachers’ fault, not his. In an era where Accountability is the hallmark of education policy, King was quick to refuse any accountability for what happened on his watch. These days, the ones at the top never accept accountability for what goes wrong, that’s for the “little people” like teachers and students, not for the bigwigs. No one holds them accountable, and they never accept any. None of them ever says, as President Harry S Truman did, “the buck stops here.”

So this is the reason that even a stupid, pointless story like the pineapple story–so thoroughly bowdlerized that it was disowned by Daniel Pinkwater, its original author–got past the review panel. I know about this process because I spent seven years as a member of the National Assessment Governing Board and served on a committee where we read every single question that would appear on a national test. When the review committee gets the items, with questions and answers, you are told that this particular item has been thoroughly field-tested. It has appeared in a children’s magazine; it has been used in a state assessment. Here are the results with all the accompanying statistics for this item. You are also told that the publisher’s own technical reviewers approved the item; so did the publisher’s bias and sensitivity reviewers.

By the time the item reaches the teachers or external panel, it has been vetted, you are told, by many others. There is tremendous implicit pressure to go along with the judgment of others whom you assume are very professional. They all agreed it was fine. Who are you to raise a question or complaint?

Since I am by nature a skeptic, I always read test passage and their questions and answers as if no one  else had. And on more occasions than I can count, I said, “Stop. Wait. This doesn’t make sense. The question isn’t clear. None of the answers fits the question. There are two good answers,” or words to that effect.

But I  understand the social pressure, the social consensus, that discourages questioning and criticism.

And that is how bad questions get onto standardized tests, and why the Pineapple question was not the first and will certainly not be the last to slip past the review panels.

The best remedy for this problem is to publish the questions and answers when the tests are finished. That way, everyone can see them. After all, as Mayor Bloomberg and Governor Cuomo and Secretary Duncan often remind us, when speaking of teacher evaluation ratings, “The public has a right to know.”

Since the tests are the linchpin of every national education policy today, the public has a right to know if the tests are fair, valid, reliable and reasonable ways to assess student learning.

Diane

Categories Bloomberg, Michael, Cuomo, Andrew, Duncan, Arne, Testing
Newer Entries »