The state of New York released some of the questions that were used on its tests for grades 3-8. Kate Taylor and Elizabeth Harris of the New York Times wrote about a question on the third grade test that more than half of the children got wrong. When the author of the passage was asked the same question, he got it wrong. After he heard the “right answer,” it made sense to him.

The East African fable goes like this: A man frees a snake that is trapped between two rocks, and as a reward the snake gives him a charm that will allow him to hear what animals say, but only if the man keeps it a secret. The man betrays his new power by giggling at the things he hears, arousing his wife’s curiosity. He eventually tells her about the charm, and it stops working.

This story, which was included on this year’s New York State third-grade reading test, is easy to read. But a couple of the questions that went along with it on the test were trickier, stumping many third graders and, perhaps, even a few much older readers.

Some of the questions were relatively easy, others were “hard.” But “hard” seems to mean that they were confusing, poorly written, and made no sense, neither to children nor to many adults.

Peter Afflerbach, a professor of education at the University of Maryland and an expert in reading assessments and comprehension, said he considered the questions to be a mix. While some of the simpler questions seemed acceptable, he said, the more complex ones could sometimes be confusing.

“A really important guideline for item-writing is you never want the prompt to be more complex than the text the child actually read,” Dr. Afflerbach said.

Dr. Afflerbach was troubled by the third question: “How does Niel add to the problem in this story?” The correct answer is B, that “he laughs at what the animals say.” But the entire premise of the question might not make sense to a child, he said.

“As a third grader, I’d be thinking, ‘How is a magical charm a problem?’ ” Dr. Afflerbach said.

The reporter tracked down the writer of the passage. “He was initially stumped by the same question Dr. Afflerbach took issue with, but when told the answer, he said it made sense.” 

Probably the third graders had the same response. When confronted with the question, they were stumped. And they picked the wrong answer. If they had been told the right answer as the author was, then it would make sense. If the author of the passage can’t understand the question or the answer, on first hearing it, why should third grade children? This is a “gotcha” question, unfair on its face.

If

Prob

P

Some of the question

Some of the