Testing expert Fred Smith worked for the New York City Board of Education for more than a decade. Now retired, he assists parent groups understand what the testing corporations are doing.
In this post, he reviews the items released by Pearson (via Questar) to New York. 75% of the test items on the ELA were released. He wonders, why not all of them? We the taxpayers bought them, why not release them to see what we paid for?
He goes through specific test items to show their flaws.
This is a useful review of what the testing corporations are doing.
Diane Ravitch's blog 
” Item statistics have not been made available. This kind of overarching data based on how the test population performed on each item is useful to researchers, analysts and anyone interested in seeing how the items functioned. The items are the bricks that go into constructing the exams and on whose strength and quality decisions about children, teachers and schools have come to depend. ”
I strongly agree with this. I recall seeing that on past tests, there were basic reading comprehension questions that 90% or more of the students would get right. And some that very few would. It’s important to analyze that because if most students are answering the comprehension questions, it speaks to their education probably being much better than the test companies would have you believe. If they are missing the ambiguous ones or quite advanced ones, it is fine if the purpose to to distinguish high performing students who can read and analyze above grade level, but it is not fine if the fact that many students missed those questions is a sign of them being “below grade level” because the powers that be want to prove that schools are failures.
LikeLike
Got it Quick read on my part. Excellent explanation on your part.
I just don’t see it as the way such a problem would be approached in the real world.
So class, I have ordered 56 pizzas for the year end party. I have asked the pizza parlor to cut each into 17 thin slices, thinking that each student could get one slice. Will 17 slices per pizza provided enough portions so that each of our 1,000 students each gets one slice?
Who would ever phrase this problem as 56 divided by 1/17?
Wouldn’t it be more properly phrased as 56 X 17?
Or wouldn’t the more common example be: We will have 896 students attending our year end pizza party. We can only afford 56 pizzas; how many slices per pie will we need so that each student gets one slice?
To ask an 11 year old what 56 divided by 1/17 = without any context becomes tricks with numbers and is a nonsensical waste of time for most children. The human brain craves knowledge and skills that are useful and logical. This is where math gets stupid and looses most of its audience. Its like the shop teacher having students practice balancing a hammer on their finger because they can – and never using it to drive a nail to make a bird house.
LikeLike
“IGot it Quick read on my part. Excellent explanation on your part.
I just don’t see it as the way such a problem would be approached in the real world. ”
I don’t either. All you have to do is look at the convoluted problem I tossed out. Maybe we need an actual math teacher to come up with some clever way to create a word problem that would make sense and have relevance in an eleven year old’s world.
LikeLike
I sat down to look at the 5th grade math questions yesterday during my prep and ended up working through lunch and “taking” the whole test. I’m a fairly decent math thinker and have successfully taught scores of 5th graders the underlying concepts. But I had trouble with some of the questions.
Sure, I could have looked at my teaching notes, and spent a little more time puzzling through the difficult problems, (they’re challenging but not impossible questions) but the point is, our students take these tests under pressure (even if they’re un-timed,) and the answers just shouldn’t be so elusive. I understand throwing in a few questions to challenge the stronger students, but there were many challenging questions that frankly didn’t make much sense if the goal is to see how proficient students are, overall, with the math standards. And if the goal is to test the supposed “effectiveness” of teachers, then this post could most definitely be even longer…
Question # 9 requiring students to know the “trick” of how to divide fractions is the one that sticks in my craw the most. The State’s own declaration of what Shifts in Focus the CCSS are supposed to lead to include “deep comprehension” of topics and not merely learning/teaching “tricks.”
While it’s true that a truly astute student might know or remember that a whole number divided by a fraction is a whole number, including a problem like “56 divided by 1/17” is ridiculous. (For those not familiar with fifth grade math, this is the first year they work with division of fractions. It’s an incredibly complex concept to understand with “deep comprehension” so to create a test problem like this one pretty much forces teachers to teach the trick of “Keep, change, flip.” Then the answer is simple. And math short-cuts and tricks will surely remain the safety net for teachers and students. Maybe that is all for the best…)
Maybe I misunderstood the intention of all the new math standards and the underlying focus on “deeply understanding mathematical concepts.”
What do I know? I’m just a 5th grade teacher.
LikeLiked by 1 person
Could you help me understand “56 divided by 1/17” and change it into a practical word problem. I am having trouble envisioning a real world application where 56 “things” would ever have to be divided by 0.058823529. Thanks.
LikeLike
Something like if I divided each of 56 barrels/pallets/… of surplus vegetable oil/cheese/cornmeal/… into 17 equal parts ( someone has decided that dividing each container into 17 pieces would provide a reasonable amount to each recipient), how many people could receive an allotment? I taught my struggling students to think of division as asking how many pieces of a certain (fractional) size were contained in the number to be divided. It helped not only them but me, too! 🙂
LikeLike
I’ve heard that only 5/4ths of school reformers understand fractions….:)
LikeLike
2old2teach
In this math item for 11 year olds, 56 is not being divided into17 equal parts, 56 is being divided by 1/17th!!!!!!!!!!
LikeLike
Read what I wrote again, Rage: “…if I divided each of 56 barrels/pallets/… of surplus vegetable oil/cheese/cornmeal/… into 17 equal parts…” Take each of those 56 barrels and divide each one into 17 parts. I’m not saying the problem would be good for 11 year olds .
I used to hate word problems written about things I had no interest in ever knowing! 🙂 “I taught my struggling students to think of division as asking how many pieces of a certain (fractional) size were contained in the number to be divided.” So each unit (1) in 56 contained 17 pieces/parts. 56 of those units would contain 56 x 17. Getting them to understand the reasoning before someone invented a word problem was critical. They could grasp the concept if we drew 56 rectangles and cut each one into 17 pieces. I was teaching 7th and 8th graders who got left behind before. they wouldn’t have liked my off the top of my head word problem any more than you did.
LikeLike
2old2twach.
See above for my misplaced response.. Thanks.
LikeLike
rickbobrick56, I looked for a misplaced response, but there isn’t one.
LikeLike
The fact that this one revealed test question has generated questioning and confusion amongst adults on this blog is a small hint at what our students face. (And points out the critical importance of releasing questions. Everyone needs to see concrete evidence of what teachers have been saying for years – These tests are not to be trusted and need to be changed.
There just isn’t any logical place where the fraction problem I mentioned would make sense in an honest testing of student and teacher effectiveness, unless you just wanted to see if students had learned the trick of “keep, change, flip.”
I’ve questioned the SED on many occasions, including Commissioner Elia in person, about how teachers are supposed to teach for deeper mathematical comprehension given the amount they must learn in a year, and the too-often non-sensical questions they face on State exams.
The Commissioner didn’t understand my question and was unaware that the CCSS came with a directive to “Shift Focus” in teaching. The people who work in the SED offices just cite pages in the engageny curriculum to show me where the topic is taught. But they can’t give a real reason why that particular question can be considered legitimate on a 5th grade math test.
OY, I say. OY.
LikeLike
I feel your exasperation in dealing with NYSED. Elia is just another empty suit. She is less than worthless and would rather go down with the ship than help board the lifeboats. The Regents Reform Agenda of Andrew Cuomo is doomed, but she is too insulated to have a clue. Cuomo no longer cares about his assault on NY public teachers, parents, and students as his political ambitions shift toward HRC and DC.
LikeLike
Hey, at least some of the questions are released. Could be worse. Utah has NEVER released a single question from ANY of its standardized tests–that’s going back 15 years or so. So we have NO IDEA what is asked on the tests.
LikeLiked by 1 person
My own school district talks about “unpacking” the scores the children in the district received. Without the detailed information Fred Smith points out is necessary to do such an analysis, I wonder what they are using to “inform instruction.”
LikeLiked by 1 person
We just had a chance to completely analyze test scores and student responses on the completely transparent NYS science exam. We had all the detailed information that Fred Smith would want. Unfortunately. most of deficiencies identified have nothing to do with curriculum, pedagogy, or classroom issues. The root problems include, high rates of absenteeism, student apathy, lack of parental support/family dysfunction, and accrued skill deficiencies (esp. in writing and *reading). Combine this with the ridiculous demands to test every child regardless of cognitive damage or limitations and to have first year ELLs taking tests that they cannot read. Now factor in overall testing fatigue and a one day, Monday morning exam that requires way too much reading stamina, while covering four years of science instruction; and we sat there wondering . . . What’s a teacher (or school) to do?
*e.g.
Q: Which two animals in the food web is a first order consumer?
Student response: grasses and trees
LikeLike
I feel for you. I imagine these are the conclusions we would have drawn in the last high school in which I taught. My local community is more like John King’s suburban targets whose children are not as smart as their mothers think they are. Unfortunately, too many of those “mothers” in my community still get hung up on numbers and hard data. The kids are benchmarked to death; the state is going to require that 30% of a teacher’s rating be based on student growth scores. It is so sad to see what used to be a strong progressive district destroy its long history of progressive thought. It is not good to come from the state from which both Arnie and Barack emerged.
LikeLike
“. . . are first order consumers?”
Does anyone really think that a 14 year old does not know that trees and grasses are plants – not animals? Responses like this are the residue of test burn-out as much as anything.
LikeLike
But numbers don’t lie! (snark alert)
LikeLike