Now that so many of our policymakers consider standardized testing the ultimate measure and goal of education, it is a good time to step back and remember how error-prone these instruments are.
Fortunately there is a new website that is collecting truly stupid test questions.
In my seven years on the NAEP board, I saw many questions with two good answers or none.
Be sure to read it and submit your own entry.
Diane Ravitch's blog 
This might be rather odd: “I saw any questions with two good answers or none” Editing needed, Ms. Ravitch?
John Gulino Oakland Unified
John–
I read “any” as “many,” as I’m sure most people did. Let he among you who is without the need to be edited cast the first stone… 😉
Shu
I fixed the typo. Sorry.
Forgive the typo. Fixed it. Sorry.
I believe she was simply checking readers closure skills (m)any.
Can’t wait to read these! Unfortunately, teachers in Washington state are not allowed to look at the tests.
Sent from my iPhone
And teachers in Maryland sign a confidentiality form that says we can’t share questions. Sorry. I see poor questions all the time. I used to report them, but when they keep turning up I gave up.
I am not losing my job over this. I would never give out test questions, even if they are silly.
I agree…no disclosure of questions…certainly wouldn’t want anyone to wind up on “Double Secret Probation!” ~ Dean Wormer
How about this from the SAT 9 (about 12 years ago): “Write as many mathematical sentences with the whole integers 0-9”?
OH NO, how UNETHICAL of me to have read the test that I was forced to give. And that was just the tip of the iceberg as far as bad questions on that test. Minimum of 25% problem rate with the questions for each section, one as high as over 50% problem rate.
There’s a reason they don’t want the test givers to read the test, folks!!
“. . . mathematical sentences as you can. . .”
So true, Duane! Unfortunately for them, special ed. teachers DO read some of the tests aloud (a script is even provided, so you don’t have to read over a student’s shoulder!), such as Math and Science sections. I cannot specifically recall questions (I retired in 2010), but I most certainly DO remember that a number of Math questions throughout the year had either-1. more than one correct answer, or-2. NO correct answer. Additionally, some of the Science questions were so convoluted as to be unanswerable. Finally, everyone in the school (middle school, 6-8) was giving test prep from Pear$on materials (they are the creators of the IL State Achievement Tests-I.S.A.T.s), so EVERYONE was able to read poorly constructed reading samples, math samples & questions/answers. Also, the Extended Responses that were scored on the samples were such that the Language Arts Teachers could NOT figure out just how the scorers arrived at their scores. Again–I suggest you read Todd Farley’s book, Making the Grades: My Misadventures in the Standardized Testing Industry.
Ditto THAT!
I read this test question scenario in one of my texts, and apologize for not having a citation:
If A is similar to B, and B is similar to C, then A is similar to C. True or False? The “correct” answer is stated as being True. However, a creative child stated the following: If A is a green apple, and B is a green car, then A is similar to B in that they are both green. If B is a green car, and C is a red car, then B is similar to C because they are both cars. A green apple is not similar to a red car, so the answer is False. This creative thinker would answer the question “incorrectly.”
Seems to me that a logical thinker would be even more likely to get it wrong. Syllogisms don’t work well, if at all, when the categories are linked through analogy.
Exactly the problem…testing today is not designed to evaluate creativity but, rather, to trest one’s ability to spit back that which was taught to be tested. Unfortunately for us all, that process is the antihesis of real learning, Too many who should know better continue to fail our children.
One of the things that bothered me about NAEP questions, outside of the reading test, was that every other subject was a reading test. Many of the math questions are tests of reading. The student who is a poor reader but gifted in math will do badly and not be able to show her proficiency in math.