Anna Allanbrook, principal of the Brooklyn New School, wrote a letter to the families of the school expressing her view of the new ELA tests.
Yes, they were harder, because so many of the multiple-choice questions had more than one answer. This is simply very bad test design. Test questions should have one answer, not two or three. This is confusing to students and serves no useful purpose.
The tests were “harder” because they spanned three days. One teacher wondered why it took three days to figure out a student’s reading level. A reason the tests were longer, though not the only reason, was that they included field-test questions, and the students could not know which was which.
For the students who are struggling learners and for special education students, the tests were harder, and they were made aware of how far behind they are.
And then she asks the hardest question of all: At a time of cuts to basic services and shortages of supplies, why is there always plenty of money for testing?
Diane Ravitch's blog 
At what point should a student be made aware of how far behind they are?
LikeLike
How about instead of focusing on what kids can’t do, let’s start focusing on what they CAN do. I guarantee you, kids who struggle are PAINFULLY aware that they struggle. They don’t need it thrown in their face any more than what the experience on a daily basis. Just because a child struggles or has special needs doesn’t mean their CLUELESS as you seem to assume.
LikeLike
I totally agree, Hannah.
LikeLike
Hannah, I am using the wording of the original post by Dr. Ravitch. It is Anna Allenbrook who criticizes the ELA tests for making struggling students ” aware of how far behind they are.” It is her assumption you are criticizing, not mine.
Nor am I focussing on this. It is, however, one of the criticisms of the ELA tests given by Anna Allenbrook so I think it deserves some attention. I don’t think there are easy answers to this, and I struggle with this question both as a teacher and as a parent.
LikeLike
Oh, pardon me for not choosing my wording appropriately. I meant to criticize you. You seem intent on splitting hairs about the words people use as opposed to the intent behind them. Neither Diane nor Anna need me to defend them… they are far greater wordsmiths than I. I should know better than to feed the gremlins after midnight.
LikeLike
You do not think Anna Allenbrook meant this point as a criticism of the ELA test?
LikeLike
Hannah,
Don’t bother. It just goes round and round and on and on with no resolution. Eventually it will end with a story about his son.
LikeLike
Yup… no more feeding gremlins after midnight.
LikeLike
This question will NEVER be answered.
LikeLike
“Why is there always plenty of money for testing?”
The cake is a lie.
LikeLike
I helped proctor these ELA pilot tests in my district. While they are not perfect, and they are longer than I’d like, I found them to be better than I expected. As for questions having more than one right answer, I find that to be challenging but not necessarily bad test design. “Which of these statements is true? More than one statement may be true.” Why should test questions have one only answer? This is only confusing to students because multiple choice tests have historically asked for one answer. “Which one of the statements is true? Choose the best answer. ” Allowing for more than one correct answer choice requires additional thinking (less guessing) which I think is a useful purpose.
LikeLike
When a test question has more than one right answer, students should get credit for choosing every right answer, not just the one favored by the test maker.
LikeLike
I agree with that. I guess that’s a criticism of the scoring more than of the question (or question format) itself.
LikeLike
I’d be more inclined to think like in the example I give below, that if a student picked the ‘most precise’ of 4 correct answers, they got lucky and guessed right.
LikeLike
Mrczarnecki,
How would you feel about getting canned on account of kids missing these ” multiple correct answer” questions?
Personally, I wouldn’t care for it.
LikeLike
Then let’s make these tests public after they are administered so all of us can judge.
LikeLike
mrczarnecki,
Did they pay you extra to proctor the pilot tests? If not, do you always give your time (and/or the district’s time) away for free? If you proctored these tests how can you know what is on them? Aren’t the proctors instructed that they cannot read the questions so as to not be able to subtly help students? And if not then is not the test a little less than “standard”?
Excuse me for saying, but you are a pawn in a very harmful game called educational standards and standardized testing. Wilson has shown the myriad errors involved in the production, giving and disseminating of the “scores” that render said process completely invalid*. Therefore any conclusions drawn from the process are, as Wilson states, “vain and illusory” and mean absolutely nothing in regards to logical thought and conclusions. The only true facts about these standards and standardized testing is that they cause serious harm to many students and that they cause great profits for those who make them. Great to see that you, mrczarnecki, are a willing puppet for the edudeformers-NOT!!
LikeLike
Sorry, forgot to add the Wilson link: “Educational Standards and the Problem of Error” found at: http://epaa.asu.edu/ojs/article/view/577/700
LikeLike
The EdReformers preach about Quality in Education until we want to ______, you know what. But, we keep hearing reports from teachers and administrators about the quality of the test items, the multiple answers per question, poorly written, irrelevant passages, boring passages, field test items, etc. Yadayadayada! We point those things out, but we need to step up our concerns and our tactics!
Writing excellent test items is critical if our kids are sitting for hours, teachers are judged and fired, and schools are closed.
Professionally, we behave as if we still respect the process and see a purpose to this madness. We don’t! How can these testing companies continue to publish SUCH SH*T, and we follow the administration of tests, posting results, remediate, and respond to it as if it HAD ANY MEANING OR WAS OF QUALITY – to the letter? BECAUSE THEY CAN!
We have too much evidence of this junk science. I wonder if Pearson has large groups of prisoners writing these questions in exchange for cigarettes in our thousands of jails in the US? Or, they outsourced this work and some folks half way around the world are writing these questions. Makes we wonder. Remember, it is NOT about kids or Education!
We need to step it up…because they are! Using our kids and teachers for financial gains and sadistic pleasure. We know what good test items are, and what they should measure.
LikeLike
The problem is many of these exams are prepared by business folk, not educators. Teachers are trained in various methodologies and the coursework includes test creation. They learn about such things as distractors, where an answer is MOSTLY correct except for one key piece of evidence. However, teachers are usually excluded from all of this big business bs. Hell, Bill Gates and the whole Common Core nonsense was decided upon by 60 educational reformers; only ONE was a classroom teacher!!! No wonder the tests are flawed.
LikeLike
Why THE $$$$$$$$$$$$$$$$ for Testing..
In bed with the Giant Book Company.
Book Co has a book and word for word…ccss was taken from that book…..
We all know the Giant..
LikeLike
Anna
You are brave…
Hope they do not fire you for asking questions as they do in so many states.
LikeLike
I had the same the same observations about the SBAC tests and wrote to the Superintendent of NC asking why we want to roll out tests obsessed with having students identify ‘most precise answers’ which SBAC heavily promotes in its sample items, and now practice test for ELA. She responded to me (surprisingly but I’m encouraged to see they are concerned). She stated that they are still in NC evaluating the tests (they will recommend rollout after the dep of pub instruction see pilot/field test results with legislature approval), and that she agreed this was a bad test item, and that all tests like the ACT/SAT tend to have unfair questions. Here is what I wrote to her and the board of ed:
“I support ending ridiculous high stakes testing where it seems Smarter Balanced is patting itself on the back for it’s ‘clever’ trick questions like the ones my son, the schools and our districts will be evaluated on when smarter balanced rolls out tests in NC:
See this sample SBAC ‘authentic assessment of student learning’ Language Arts question for the tests rolling out in NC unless the legislature stops it:
Read the sentence from the text. Then answer the question.
“Nanodiamonds are stardust, created when ancient stars exploded long ago, disgorging their remaining elements into space.”
Based on the context of the sentence, what is the most precise meaning of disgorging?
a. scattering randomly
b. throwing out quickly
c. casting forth violently
d. spreading out widely
I asked my 6th grader what he would answer. He said spreading out widely since as he put it, the universe is expanding so a star that explodes would spread its particles out widely in space. I told him, well SBAC thinks you need improvement in reading and cognitive thinking. The correct answer ‘seems’ to be ‘casting forth violently’ based on Merriam websters definition of disgorge, and I’m still scratching my head wondering why it wouldn’t be ok for a 12 year old to answer ‘throwing out quickly’ as that too is implied in the meaning of disgorge. Honestly, I’m still not sure.
The tests are replete with what will be just a bunch of confusing and stressful questions for children that will never shed light to educators how well a student is learning. I told my son that his thought process was well grounded in science, and that he’ll likely be faced with lots of questions like this and his approach should be to do exactly what he did – give a well thought answer he knows he could defend. Let the schools say he failed…Einstein’s theory of relativity proves he had every certain reason to understand that an exploding stars spread widely through space…and I don’t value that he couldn’t precisely define disgorge in the context of the passage. I couldn’t either!! LOL!
What a travesty and waste of money. If anyone one can shed light on what the ‘most precise’ answer is and confirm whether SBAC gets a kick out of these trick questions at the expense of our tax dollars, I’d be very grateful!”
LikeLike
A OR B OR C OR D…
ALL ARE CORRECT
LikeLike
“To eject from the throat or stomach”
“pour forth; discharge
Oxford American Pocket Dictionary of Current English
‘to vomit”
to pour forth (its contents); empty (itself)
Webster’s New World Dictionary
I think that I am going to disgorge my remaining supper.
LikeLike
@Lolita:
As a 20-year language teacher, all I can say is that the sample question is:
a.) poorly designed
b.) deliberately misleading
c.) not diagnostically useful
d.) all of the above
Okay, what’s the correct answer? This is a test…
Oh yeah, AOTA questions are bad test design too. *sigh*
LikeLike
Isn’t a multiple choice item with more than one correct answer by definition invalid?
LikeLike
I use the multiple response format in all of my classes and prefer it over the traditional multiple choice format. However, if student have not been exposed to or are familiar with the format, it is very challenging, and I dare say, too much so.
MR offers many benefits. It has higher assessment density, reduces guessing for the one “right’ answer, requires students to evaluate every choice for its veracity and appropriateness. Furthermore, students actually do better on these items in my class because I award points for both choosing the desired response as well as NOT choosing undesired responses. In life, I believe that it is equally important to know when something is right and when it is wrong. As a format, MR items should really be thought of as a set of T/F questions that are happen to be grouped together.
As is true with any item format type, there is always potential for abuse. Items can be made more difficult by A) requiring deeper understanding/higher ordered thinking skills or B) making the item confusing and deliberating tricking the student. As posted in one of the replies above, one way to make a question confusing is to make the choices ambiguous by splitting semantics. Knowing test makers, instead of testing for learning and critical thinking skills, they are testing for how well students can game the exams. They can do this whether using MR or MC item formats. When doing this with MR, it is even more frustrating to students, because they have to game every choice and not every 5 or 4 choices due to the higher assessment density.
LikeLike
“MR offers many benefits. It has higher assessment density, reduces guessing for the one “right’ answer, requires students to evaluate every choice for its veracity and appropriateness”.
Horse manure. Any teacher who uses only MG* test format isn’t worth the salary that they are paid. It’s probably one of the worst “educational inventions” that has ever come around-a completely bogus format.
*multiple guess
LikeLike
She asks “At a time of cuts to basic services and shortages of supplies, why is there always plenty of money for testing?” The answer is simple – people are making profits from that, and that is their concern, not the well-being or real learning of the students. Yet again we are privatizing what should be the common good.
LikeLike
Agreed, I recently posted on my own blog about why we keep deprofessionalizing and demeaning the teaching profession. We keep talking about holding teachers accountable and firing “bad” teachers. In my opinions, no other profession has become more regulated by the government than teaching. And it all comes down to the fact that teaching doesn’t make money for companies the same way doctors make money for pharmaceutical and medical equipment companies or engineers make a product that can be patented and sold.
Good teaching means less testing and micromanagement. That means less money. It is imperative to the corporate reform movement that teachers be painted as incompetent and uncaring.
LikeLike
i think the regulation of public school teachers is primarily the result of 1) an attempt to prevent the public education system from providing patronage jobs to local politicians and 2) the inability of most families to choose a teacher or a school in the public school system.
LikeLike
“why we keep deprofessionalizing and demeaning the teaching profession.”
Well, when one insists on using a bogus format* for assessing students perhaps they deserve to be as demeaned as the teaching and learning process is by using said format.
*multiple guess tests
LikeLike
I don’t know what a MR item is as opposed to a MC item.
LikeLike
Lolita,
I recently went to a department chair meeting for high school English with at least 20 English DCs in attendance, and we were shown a 9th grade SBAC question asking the student to define countenance, which was used twice in a reading passage in slightly different ways. No one could agree on the answer, and the ELA coordinator running the meeting had to send her assistant to get the answer key out of her office. I ended up having the right answer, but I wasn’t confident about it, and the other explanations teachers had seemed extremely plausible. I do practice SAT and AP questions with my students all of the time, and I have never seen a poorly constructed test item like this one. I have zero faith that these tests are valid. The writing prompts are also poorly constructed. They read like AP Language prompts, but are more confusing and harder. I am extremely resentful my school district will be wasting money on these tests after giving us a pay cut, including not honoring their commitment to increase pay for teachers who obtain master’s degrees. I will be opting my daughter out of these tests. I would rather she sit in the library for three days and read.
LikeLike
So, I am not the only deficient SBAC test taker…lol! I am so sorry you have to attempt to ‘prep’ this generation for something this ridiculous and meaningless.
If these tests roll out, I am likely to opt my kids out of them too…I don’t believe that is legal in NC, my state, as the courts have upheld testing as something that is vested in the right of the child (to know where he/she stands relative to peers and to know if he/she is receiving a sound education) so what I would do is have them just take the Iowa instead. Let them be done in 45 minutes of reading and 45 minutes of math, check off they are on grade level, and move on to something more interesting.
LikeLike
Reg… That’s one of the bests posts I’ve read in a while 🙂 I miss teaching high school English, I really do, but I’m glad I’m not doing it now. With all this swirling around, instead of simply blogging my ire I’d probably have a full-on meltdown in my classroom.
LikeLike
What is troubling is all the emphasis on testing and not on retention or application. Teaching to a test and teaching are two different things. I would prefer students to be life learners, not lab rats working feverishly on test-taking skills to complete exams prepared by educrats not educators. The ability to imagine, create, embellish… all are forsaken as they have no place when the answer is one of five possible items to be bubbled in. We are not raising a nation of critical thinkers but a nation of pencil pushers.
LikeLike
Ken is right, just follow the money. It rings true in criminal investigation, call it motive. We are on the slave block, we will have to ransom ourselves by exposing these secret tests and educating the public so that they know that these tests will become their children’s shackles.
LikeLike
It is imperative that the public be educated as to all that is happening in the false name of reform. Most of the public, including sincere and aware people, have no clue about all of this.
Media is hiding under their collective beds so as to not become embroiled for they are reliant on the billionaires who own them for their livelihood.
LikeLike
That question about the money spent on questioning keeps banging against all sides of my brain. Altogether, given the evidence against pervasive (& punitive) standardized testing AND standardized curricula, wouldn’t students be better off if all the money being paid to Pearson & their ilk went straight to schools – even if more dollars did “nothing else” but to shrink class sizes?
LikeLike
It’s about time NYC principals told the truth about this test. Teachers weren’t even allowed to look at the test unless a child raised their hand about a problem. Even then a teacher could only guide the student back to the directions, but it did give them an opportunity to see what was confusing the student, and it also confused the teacher.
We need these tests to be made public after they are administered. By June every teacher and parent should be able to see the test. Last year the NJ ELA asked 3rd graders to share a secret. To many parents, that was an invasion of privacy and an inappropriate question. The Pineapple story was also deemed confusing, even by the author of the Pineapple story because of the way it was edited.
What is Pearson and the states protecting?? Why are teachers being fired based on a test they cannot see for themselves?? Were are their constitutional rights??
LikeLike
“We need these tests to be made public after they are administered. By June every teacher and parent should be able to see the test.”
This, to me, seems eminently reasonable. NYS posts their Regents exams to the NYSED website within a few hours of their completion (as well as answers and a scoring key and rubric). Why can’t Person et al do the same?
LikeLike
Andrew,
That answer is simple. Now that teachers’ scores are tied to the test, we have no recourse to defend ourselves. We know these tests have some major flaws, and by not releasing them, both students and teachers suffer while Pearson is able to cover up their misdeeds. Pearson’s has a reputation of spreading money and trips around to get these contracts. They are using their influence to have their tests hidden from the public. And it’s not just in NYS. Other states are also not allowing teachers to see these tests. If we do and speak out against an unfair question, we run the risk of losing our jobs.
LikeLike
We need these tests to be made public after they are administered. By June every teacher and parent should be able to see the test.
I have to agree with this. If these tests are intended to help teachers improve their teaching, and/or help to assess the student’s weak and strong areas, how does hiding the test help?The information needs to be provided to teachers/parents in a swift and complete manner so that they could use the material to actually help an individual child or so the teacher can improve their methods to help all their students.
It is time we began asking NEW questions ourselves:
If we are spending this testing money to improve an individual student’s education, does it do that?
If not, why ARE we spending it, and who benefits from it, and should we continue to do it?
To assess if the tests actually improve education, we have to ask several other questions.
Is it being used for the purpose for which it was designed? (not to rank schools, students, or teachers)
Is it appropriate to the grade level/ability level it purports to test? (ESL and SpEd, take note)
Is it pedagogically sound? (Are there pineapples involved?)
Is it structurally sound? (No twist questions. No MR questions hiding in a SR set-up)
Is it correct? (Face it…we’ve seen typos in books, and incorrect answers in books. Are we sure that the tests don’t have typos and incorrect answers?)
Is it properly graded? (Essays read by temps, anyone?)
Is it administered in an appropriate manner and setting? (45 minutes or less per test, with breaks, and with adequate heating/cooling/seating/lighting?)
Are the test results returned in a timely manner? (Can I help my student improve, or are they long gone from my class?)
Are the test results returned in a manner that allows the teacher to see which specific topics need improvement? (All my students missed the question on Pythagorean’s Theorem vs. The number of questions the students missed.)
Are the test results returned in a manner that allows the parent/student to see which specific topics need improvement? (Here is your essay with marks for punctuation, grammar, spelling, etc.)
Is there any way for students to find out the correct answer to questions they have about the test? (The actual method for teaching kids…help them find the answers to questions they have themselves!)
Is there any way for parents/teachers/students to contest the results of the tests?
LikeLike
janietta,
Any standardized test by definition is not valid. Noel Wilson has proven as much in “Educational Standards and the Problem of Error” found at: http://epaa.asu.edu/ojs/article/view/577/700 .
So all your questions and concerns are, in reality, for naught as they are more mental masturbation concerning whether or not these tests are valid, which they are not.
Get to the heart of the problem, that educational standards and standardized testing have so many sources of error in the construction, giving/taking and disseminating of the results that renders them completely invalid. The basic problem is that standards and standardized testing attempt to quantify a human activity-the teaching and learning process-that cannot logically be quantified as the teaching and learning process falls in the realm of an aesthetic or quality of human interaction. Quantity by definition is a subcategory of quality and therefore logically cannot be used to define/assess/evaluate quality.
LikeLike
There’s always plenty of money for policies bought and paid for through campaign donations for politicians and high paying jobs for the politician when they leave office (and even for their family while the pol is still in office).
The quality of education or any other public good comes in a distant second to following the money’s orders.
LikeLike
Today’s NYTimes has 2 excellent op-ed pieces on the pitfalls of CC. One “The CC: Who’s Minding the Schools?” references Diane and the disparity it will cause on the “haves and have nots”. The other, “No Learning Without Feeling” is just from the heart of an English teacher.
The latter did not allow for comments just as the NYTimes editorial on the teacher evals did not allow for comments. Other editorials, like the one today on Immigration, have a comment section. I think the NYTimes knows when to expect blow back and doesn’t want facts presented that can prove embarrassing for their IMHO convoluted opinions on education.
LikeLike
The Learning Without Feeling was interesting. i particularly liked this excerpt:
“The writers of the Common Core had no intention of killing literature in the classroom. But the convenient fiction that yearly language learning can be precisely measured by various “metrics” is supplanting the importance of literary experience. The Common Core remains neutral on the question of whether my students should read Shakespeare, Salinger or a Ford owner’s manual, so long as the text remains “complex.” ”
I blogged recently about how literature is being sucked out of the high school ELA programs (http://askingquestionsblog.blogspot.com/2013/03/the-canon-and-other-instruments-of-war.html), replaced with trendy topical non-fiction or simply teaching to the standardized tests. I defend the canon, which may be a bit of a conservative stance for many of this blog’s readers, but the larger principle of Great Works and Great Literatures and reading with “feeling” is a commonality.
LikeLike
Don’t just say that test questions should have one answer like it’s a fact. That is purely your opinion and students have been trained since they were 5 years old to find the correct answer even if they don’t understand the concept. That’s what happens when your multiple choice questions only have one answer. And that’s only one type of smarter balanced question. Why didn’t you mention the performance tasks? They don’t match your bias so you chose to ignore them?
LikeLike