A parent in New York wanted to see what the testing experience was, so she went to the State Education Department website and tried the practice questions for third grade. Whatever her initial objections might have been, what she found most objectionable was the nature of the online assessment.
She wrote:
“I had the opportunity to take a practice grade 3 math CBT today. It sealed my decision to opt my children out. It was highly frustrating and difficult to navigate. The font was very small. At the beginning there were a multitude of directions explaining all the online “tools”. Not all the answer choices always fit on the screen, so you have to scroll up and down to navigate the entire question with answers. On a two step word problem I was required to show my work. To do this you have to tap on an algorithm and then plug in the numbers. For some reason, even though I chose a vertical algorithm, the two numbers were not properly lined up, which made adding them pretty tough. Even with three adults looking at it we couldn’t fix it. The problem also required carrying, and you basically had to do that in your head as there is no way to carry the extra ten to the next column. Also, I had to enter the answer from left to right, instead of adding the ones, tens, hundreds. These tests are already flawed in so many ways, and now we are adding extra anxiety to these kids. And how will the results not be invalid? How will we know if the kid really didn’t know an answer, or just couldn’t figure out how to navigate the computer? None of this is necessary for 8-14 year old children.”
Diane Ravitch's blog 
I’m actually very happy for all these design flaws because that’s what’s making people sit up and pay attention to the validity of the tests. But here’s the thing, even if the tests were seamlessly designed for maximum ease of use, they would still be invalid.
LikeLike
They will surely get slicker and slicker as time passes.
LikeLike
Yes, that’s what worries me. Then parents (and others) will sit back and be satisfied that all is well.
LikeLike
Don’t count on it.
Computer “scientists” [sic] have had decades to develop this stuff and much of it is still crap ( and not just the tests but the computer aided “instruction” as well)
People just assume that software “coders” know what they are doing, but take it from someone who used to do software development for a living and worked with a lot of these people: that is a very bad assumption.
Kind of like assuming that the people who program self driving cars would never allow the car to drive over a pedestrian at 40 mph or smash into the side of giant (very obvious) semi truck.
LikeLike
I certainly can’t predict the future and I’ve been wrong much more than I’ve been right in the past. But if you asked me 20 years ago what technology would look like in 2018, I would have never guessed it would have gotten as far as it has. My sense is that the rate of tech change is accelerating, not improving linearly.
LikeLike
You are free to do so, but I would certainly not bet the farm (or even the chicken coop) that online testing software will keep getting “slicker and slicker.”
Every time these tests are given, it seems to be the same old story: hard to use and then they crash the day of the test.
LikeLike
Beat me to the punch with that line, dienne77!
LikeLike
I would have done the same!
LikeLike
They won’t get better over time. They will ultimately be abandoned for a multitude of flaws. Just opt out until that time comes.
LikeLike
They won’t get better over time. The tests will ultimately be abandoned due to the multitude of flaws. In the meantime, continue to opt-out. Your children do not need to be discouraged and frustrated at school and, on that level, these tests are actually harmful to them. Stay positive and opt-out.
LikeLike
Ha ha Kelly. No they won’t be abandoned – because the “technology” is a magical talisman. Kids need to know “technology” in order to “get a job” so anything that uses “technology” is therefore good. Look at the adult world – if anyone had a real choice would they still be using a crap program like Microsoft Word? It has been in use for decades, has made billions in revenue for Microsoft and is still idiotic when it comes to predicting and analyzing syntax and grammar, organizing and numbering structured documents, etc. It is simply too convenient for TPTB to deflect responsibility by basing funding and other decisions on numerical benchmarks, even of the benchmarks are meaningless numbers. The bonus is that this trains kids to follow directions from their bosses, however nonsensical without asking questions – which one must admit is an invaluable skill in most workplaces.
LikeLike
You do not need a computer to offer an invalid problem. Here is one, from a NWEA test: “A block of ice loses 1/6 of its weight each hour that it sits in the sun. If a 180 pound block of ice is placed in the sun, what will it weigh after 2 hours?”
LikeLike
Not a teacher-made test.
LikeLike
125 pounds?
LikeLike
I gave this problem to my friends. Everyone answered 125 lbs including myself, but the “correct” answer is 120 lbs. The choices include: 25, 30, 120, 125, 150 lbs, so you cannot select the least crazy answer by elimination. But, when you start thinking about the problem, both of the least crazy answers – 120 and 125 – are wrong. 120 means that the block loses 30 lbs each hour, no matter its initial size, sounds crazy to me. 125 means that the block loses 1/6 of its CURRENT weight each hour, a less crazy idea, but think again – it will NEVER MELT. This is a classic Achilles and the tortoise paradox. The issue is that the problem states a constant time interval – EACH HOUR – while in reality the melting accelerates the smaller the block of ice gets, so the time intervals get progressively smaller, and the problem converges, but this is not a middle school problem. Whichever way you look at it, it is flawed. And even if you are willing to accept its flaw, saying “let’s pretend this is a middle-school problem”, you still are going to give an “incorrect” answer.
And I see crap like this everywhere, math problems or ELA. I don’t know what kind of results they expect with problems like this.
LikeLike
“125 means that the block loses 1/6 of its CURRENT weight each hour, a less crazy idea, but think again – it will NEVER MELT.”
This seems the only sensible answer to me.
I don’t find the ultimate un-meltability problematic. In questions like these, you accept the question’s underlying assumptions for purposes of the exercise, or you bomb out. (As they say in law school, “Don’t fight the hypo.”)
Anyway, a question like that never should have two options (120 and 125) that would both be obviously true depending on how one interpreted a very debatable aspect of the question’s premise. IMO, of course.
LikeLike
Gruff: Where does 120 come from? If we assume that it looses 1/6 of its weight each hour, we have to look at this as a geometric sequence with a common ratio on 5/6 (since it keeps 5/6 of its previous weight under the literal statement of the problem). This sequence would look like 180, (5/6)180=150, (5/6)150=125, …
In order to get 120, one would have to state that the block loses 1/6 of the original size each hour, an arithmetic sequence.
I suspect that this was one of those problems that was thrown out because someone recognized that it was flawed and led to bad answers. The real question is what this question says about any individual student other than the ones who have been told they are immediately responsible to know this stuff by Friday.
LikeLike
Roy: I’m still on the 2 + 2 = 4 stage of development.
Actually, I got an A+ in geometry but never made it in other higher level math classes. They never made sense to me. I memorized forever. Glad I don’t have to be in school and get bombarded every year with what a math failure I am.
LikeLike
“In order to get 120, one would have to state that the block loses 1/6 of the original size each hour, an arithmetic sequence.”
Right, that was the interpretation of the question that leads to the answer 120. It certainly isn’t how I read that question.
LikeLike
1/6 = 30
2/6 = 60
180-60 = 120
Is that so hard?
LikeLike
If it only loses 1/6 of its weight every hour, it would never completely melt.
LikeLike
Zeno’s paradox! Or something.
LikeLike
Thank you, Dienne! But you are “wrong” either way :-]
LikeLike
I thought that was Ice Cube’s paradox
LikeLike
“f it only loses 1/6 of its weight every hour, it would never completely melt.”
Correct, based on the assumptions in the problem (of course, in reality we”d have a problem when we are down to, say, 5 molecules).
This is not Zenon paradox since there the time also decreases with distance, while here 1/6th of the weight is lost at regular time intervals.
Anyhow, the problem reminds me of the mathematics of school reform: The Achievement School District here in TN is supposed to take over schools in the bottom 5%. Since there is always a bottom 5%, ASD will take over schools forever, or at least until 20 schools are left.
LikeLike
Wait – Isn’t revealing a test question punishable by death carried out by a firing squad?
LikeLike
I think it’s drawing and quartering.
LikeLike
I think “drawing and quartering” is just for the art test
LikeLike
In Lancaster, PA, you just get shunned from the educational community.
LikeLike
Gruff,
Your question isn’t a proper state question problem. It should read:
“A block of ice loses 1/6 of its weight each hour that it sits in the sun. How long would it take a 180 pound block of ice to melt completely?”
LikeLike
Doesn’t matter what it should have said, NYCpsp. It is actually fairly typical of the types of questions on standardized test. See my above post.
Someone referenced their third grader coming home and stating that he thought that the test makers were trying to trick him on his test this year.
LikeLike
I proctored a SAT-9 test once. It was given in the cafeteria and we had more than enough proctors so I went into the teacher eating area where the extra tests were at. Every section had major flaws in the questions. In the science section I found problems with over 50% of the questions. The lowest amount of questions with errors for a section was around 20%. Not very good, eh!
My favorite though was this question from the math section:
Write as many mathematical sentences as you can using the whole integers. Write your answers in the box given.
Now there are basically an infinite number of math sentences (I had to go up to the math department to verify that meant equations), so can you imagine diligent Danielle starts to write as many “sentences” as she can, furiously writing, each one getting smaller and smaller as she realizes that she doesn’t have a lot of room. Ding ding-“Times Up. Put your pencils down, close your test booklet”. Dani has only completed 10 out of 60 questions. Poor Dani’s score on the math section looks like it came from a chimp doing the test.
LikeLiked by 1 person
I thought “math sentences” were the hours of homework that the math teacher assigned you when you talked while she was lecturing.
LikeLiked by 1 person
I am with Flerp. Given the information provided in the problem, the answer should have been 125. The first hour it lost 30 pounds at which point it them weighed 150 pounds. The second hour it loses 1/6 of 150 pounds which is 25. After two hours it them weighs 125.
A sure way to blow a test is to read into questions assumptions that the question does not explicitly state.
LikeLike
“A sure way to blow a test is to read into questions assumptions that the question does not explicitly state.”
Although these days when they like to claim they are testing critical thinking,…and as we all know that process of thinking critically will always lead to thinking that reaches the same conclusions.
LikeLike
The real problem is that the person who made up the question made assumptions that they did not explicitly state as part of the problem.
And the funniest aspect is that those assumptions were just wrong (physically) WHATEVER way one interprets the statement of the problem.
As you allude to with your sarcasm, it’s actually very ironic that multiple choice questions are given to (allegedly) test critical thinking skills because the people who think critically are going to see the ambiguity and sometimes just plain wrongness of the questions, while the people who don’t think critically will more likely just choose the (superficially) “right” answer without any question or thought.
LikeLike
The problem is typical of word problems in the spirit of Common Core: kids are supposed to do close reading, paying attention only to the extraction of the hidden math, and forget about the possible real life meaning and context of the story. In fact, trying to connect it with real life casts doubts about the whole thing, as the thread shows. This is all the more ridiculous since CC does want to connect math with real life.
The problem appears ambiguous only because it is part of some kind of standardized test. In a classroom, a teacher giving this problem could be fine, since she can clarify the ambiguity immediately, or probably she gave many problems of this sort, and the kids know exactly “what she means”.
LikeLiked by 1 person
Great point. The intent of a question is knowable with a teacher-written exam.
LikeLike
“The problem is typical of word problems in the spirit of Common Core” — the problem has nothing to do with the Common Core, it is designed by NWEA. In fact, it was designed likely before Common Core became a thing.
“The problem appears ambiguous only because it is part of some kind of standardized test.” — The problem “appears ambiguous” because it was worded by someone who does not know math well enough, or does not care to present the problem in an unambigous way. The answer they deem correct – 120 lbs – comes from the premise that the block loses 1/6 of its ORIGINAL weight every hour. As unnatural as it may be, this is their assumption for this problem, and all they needed to do is to state this assumption: let us assume the block loses 1/6 of its original weight each hour. That is it, done deal. Without this assumption a student has to think about the situation, trying to align the words of the problem to the real world, because the problem is supposedly a “real world” problem, but it is not, it is just worded in terms of the real world, but in fact it is constructed back from the 180 – 2(1/6)180 formula.
“In a classroom, a teacher giving this problem could be fine, since she can clarify the ambiguity immediately, or probably she gave many problems of this sort, and the kids know exactly “what she means”.” — In a classroom this problem would be fine, but not because the teacher could clarify what she means, but because the students themselves could think about the real physical world, and, possibly with the help of the teacher, come up with different scenarios for different assumptions, like a simple linear melting of 30 lbs an hour to geometric progression to (maybe) limits. This problem would be an interesting one for a discussion in a classroom setting, but it is a totally invalid one for a test, especially with a multiple choice selections.
LikeLike
The biggest issue with these sorts of multiple choice problems is that the more you know, the more likely you are to question ALL of the answers and even the assumptions of the question itself (in this case the “melt model”)
I know that students are never (EVER) supposed to question the questions on tests, but those who know some physics will have a hard time not doing so because they understand that even when the ice block still consists of large numbers of water molecules, the melt rate depends on the size of the block.
Specifically, they will know that the melt rate depends (among other things) on the exposed surface area, which decreases as the size of the block decreases.
As a result, the absolute melt rate (weight lost per unit time) also decreases as the block gets smaller .The critical thing to recognize for the question is that the loss rate doesn’t remain constant at the initial value, which means the “120 lb” answer simply can’t be right from a physical standpoint because even if the block loses 30 lbs over the first hour, it necessarily loses LESS than that over the second hour.
But with melt rate proportional to the surface area, the “relative” or “fractional” melt rate (fraction of current weight lost per unit time) also depends on how big the block is, though in the OPPOSITE sense. Again, it’s not constant.
One would expect this “relative” (fractional) melt rate to actually INCREASE as the block gets smaller. (It scales as Area/volume ~=1/x if the ice block is a cube with side of length x, for example, so it increases as x decreases with melting). We again have a situation where the actual (physical) melt rate (in this case relative melt rate) does not remain constant as the size of the block decreases, which is also in conflict with the (second interpretation of the) problem, which assumes a constant fractional loss value of 1/6 .
So the “1/6 of it’s weight lost per hour” melt model itself is simply not right (not physical) ANY way one interprets it.
Not right/physical when one assumes it means “1/6 of the initial weight (180lb) lost per hour” and not right when one assumes it means “1/6 of the remaining weight lost per hour”.
So the REAL problem here is that the question itself is “not right”. It’s ill- posed (scientifically)
Of course, one can say ” You are reading way too much into this question which is not really even a physics question”.
My first response would be : “How do you know it was not given as a physics question?”
And my second reply would be, “Even if it was not intended as a physics question, some students who know physics will be penalized because they knew more than the people who made up the question.”
In my opinion, this perfectly illustrates what is wrong with many multiple choice questions. They are often ambiguous and become more (not less) so the more one knows.
And this is true in general, not just about science related questions.
LikeLike
Fun fact: after very deep 🙂 calculations, one can conclude that, according to the “law of melting” given in the problem, the last molecule of the 180 pound ice would melt 10^300 years from now, which is 1 followed by 300 zeroes. That’s much much much longer than the predicted age of the universe. In fact, the current age of the universe is 10 biilionish years which is 1 followed by 10 zeroes only.
LikeLike
And of course the most important test question:
Students face immediate testing following two-week walkout; will results suffer?
http://www.tulsaworld.com/homepagelatest/students-face-immediate-testing-following-two-week-walkout-will-results/article_7024e2df-d056-5b12-a35e-86006fc854a3.html
LikeLike
Invalid process leads to invalid results which lead to complete invalidity in using those invalid results for anything.
LikeLike
I bring up this study done by a National Institutes of Health researcher every once in a while since it proves that poor children, simply by living in a stressful environment, are not physically able to achieve. Their bodies are affected. They are living in what is equivalent to a war zone.
Why not test these kids regularly and enforce for them that they are totally incompetent? Put them in front of a computer for hours each day and eliminate teachers, social workers, nurses, food for the undernourished, etc. Blame the teachers for being incompetent and cut their salaries. Or give teachers bonuses if they start to produce. Make those teachers work. Standardized tests prove they are lazy and just want three months off with pay. UGH!!
………………………..
Stresses of poverty may impair learning ability in young children
NIH funded research suggests stress hormones inhibit brain function, stifle achievement.
Cortisol levels in saliva tend to follow one of four patterns in people’s response to stress. Research indicates that emotional regulation, self-control and academic performance are tied to the typical pattern, in which cortisol rises in response to stress and falls again when the stressor is gone.
The stresses of poverty — such as crowded conditions, financial worry, and lack of adequate child care — lead to impaired learning ability in children from impoverished backgrounds, according to a theory by a researcher funded by the National Institutes of Health. The theory is based on several years of studies matching stress hormone levels to behavioral and school readiness test results in young children from impoverished backgrounds.
Further, the theory holds, finding ways to reduce stress in the home and school environment could improve children’s well being and allow them to be more successful academically.
High levels of stress hormones influence the developing circuitry of children’s brains, inhibiting such higher cognitive functions such as planning, impulse and emotional control, and attention. Known collectively as executive functions, these mental abilities are important for academic success…
https://www.nih.gov/news-events/news-releases/stresses-poverty-may-impair-learning-ability-young-children
LikeLike
Quite interesting, carolmalaysia! Thanks for sharing!
LikeLike
Online testing is crashing all over Ohio today:
“Mandated state testing has come to a halt in at least some places in the state.
Locally, a Fairfield High School student reported that she and her classmates were taking the state-mandated American Institute of Research (AIR) test Wednesday morning when the system failed. After more than an hour of waiting, four students were able to log in but others could not. Testing is on hold.
A similar situation is reported on Twitter at an elementary school in Chillicothe where the technical difficulty with the Ohio Department of Education testing site resulted in a delay in the third- and fourth-grade testing. It was postponed to Thursday.”
They put the online testing in before they had to capacity to handle it – I’m just wondering- is this “putting kids first”? It seems like testing demands trump everything- like kids come last.
Maybe the state of Ohio should stop taking orders from national ed reform lobbyists and actually do their jobs, using their own judgment. They know or should know the capacity in these schools. What was the rush to go to online testing? The kids don’t prefer online tests- the systems are cheap, rushed junk and hard to use.
If they’re going to test them constantly they could at least give them decent tools.
LikeLike
What has happened to high school students? In my day, we wouldn’t have hung around for an hour of computer difficulties (and that’s in the days of Trash 80s, so we had a lot of computer difficulties). Half the kids would have been in the parking lot smoking or headed for McDonald’s. Even the “good” kids would have been hanging out in the cafeteria reading a book or in the band room shooting the, er, breeze. I guess it’s a stiff competition, but in my opinion, the worse casualty of the rephorm ear has been teenage rebellion. Glad to see some of it coming back to life (e.g., the walk-outs over gun safety). Need to see a lot more, especially in non-adult approved areas. Enough of compliance and obedience!
LikeLike
Crashed in Tennessee yesterday. See Chalkbeat report. State Commissioner is to report to Capital Hill today, with opposition politicians not resisting the urge to make hay by being mean to her.
LikeLike
Sorry, here is a link:
https://www.chalkbeat.org/posts/tn/2018/04/18/on-the-hot-seat-mcqueen-to-explain-latest-testing-blunders-to-angry-tennessee-lawmakers/
LikeLike
Then there is this, students have already figured out the only thing anybody cares about is growth. What does this mean for them? 1st test of the year, bomb it. Last test of the year do your best. Walla – magic growth and now you are rewarded for meeting the testing company’s algorithm for growth.
LikeLike
Your username makes me think maybe I can’t handle the truth.
LikeLike
What about the ACT or SAT? My son has taken it twice and received the same score. He doesn’t want to take it again but wants to go to college. Advice?
LikeLike
Pick a college that doesn’t use those false indicators of nothingness.
LikeLike
Duane’s right (as usual). There are SO many colleges that do not take test scores. My daughter–who was pretty carefree about taking tests as an elementary school student (I think the IOWAs or something were still being given then, not the “high stakes” crappier tests)–I’d told her it was supposed to be fun–like a puzzle–received high scores. But then, when older, she became really stressed out over testing; she opted herself out of the PSATs (!), because she thought it was stupid to have to take a test that meant nothing, & she refused to be stressed out more than she was (BTW, this was in 2004, as I recall). (We were very proud of her!) Anyway, she took the ACT, & was dissatisfied w/her score (she wanted to get a perfect score on the English part) &, so, took the test again (which happened to be the day after her grandfather’s funeral; we told her she did NOT have to take it but, upset as she was, she insisted). She did score a 33 (not perfect), but was pleased w.herself, so that was really what mattered.
Anyway, she needn’t have been stressed, because the university she attended (& it was the one she’d wanted to go to since she was 13)
was one that didn’t require test scores. (This was in 2005 &, marileecr, so many more schools have taken that no ACT or SAT requirement since then.)
So–tell your son to relax, & lots of luck to you & to your family in finding that perfect fit for him; enjoy the journey!
LikeLike
Well, I know someone who claim they were given a full ride for getting into top .5% of PSAT (not sure whether it is 0.5% nationally, or whatever).
LikeLike
Here you are:
http://www.fairtest.org/actsat-testoptional-list-tops-1000-colleges-univer
For most of my career, I taught in high schools where kids were poor, spoke English as a second language, and/or were the first in their families to attend college – three groups who don’t get great SAT or ACT scores. We taught them to look for colleges where the admissions process would be more interested in them than in their scores. It worked well.
LikeLike
“None of this is necessary for 8-14 year old children.”
Or for anybody. There is not a single positive thing about these tests.
LikeLike