Fred Smith, a testing expert who worked for the NYC Board of Education for many years, poses an interesting question: why were three test questions quietly removed from the Pearson tests?

He writes:

“One wonders why SED [New York State Education Department] might have killed the item. Might there be no answer? Could there be more than one correct answer? Perhaps, a higher percentage of students selected one or two confusing distractors than chose the answer SED deemed to be right? Maybe the item is biased against a certain group of students. Any of the above would give it a failing grade.”

And he raises other questions:

“So, students, what do we draw from these revelations?

A) Clearly items that Pearson claimed were vetted by review panels and experts were unrefined and no better than field test items that somehow passed muster only to flop in prime time.

B) SED’s dirty secret is out of the bag: Its performance-defining cutoff scores are set after tests are given—in this case, after the raw score distribution had been studied and truncated.

C) SED plays fast and loose with data at its disposal, withholding information from the public that paid for it.

D) Efforts to classify students and evaluate teachers that rest on such shaky grounds are indefensible and unsustainable.

E) All of the above.

“E” certainly seems like the smart choice. But we can’t know for sure until an outside investigation is conducted into how SED and Pearson have run the testing program. Parents should hold their children out of all statewide tests until SED comes clean by providing complete and timely item analysis data and is able to demonstrate that the test results are relevant to the purposes they are being bent to serve—in other words, until there are meaningful alternative assessment programs in place.

“Transparency in all matters concerning educational testing is a moral imperative. We must demand passage of revised Truth-in-Testing legislation, opening the testing process to sunshine and scrutiny, restoring its balance and something immeasurable—a level of trust in educational leadership that’s been missing too long.”

* * * *

* * * *

Fred Smith, a testing specialist and consultant, was an administrative analyst for the New York City public schools. He’s a member of Change the Stakes, a parent advocacy group.