G.F. Brandenburg, a retired math teacher and outstanding blogger,here revisits Steven Rasmussen’s critique of the Smarter Balanced Assessment Consortium’s math tests. Rasmussen was co-founder and publisher of Key Curriculum Press for many years and is a mathematics specialist. (I posted on Rasmussen’s critique here, but unlike Brandenburg, I am not a math educator.)
Here is a sample from Brandenburg:
““…the Smarter Balanced tests are lemons. They fail to meet acceptable standards of quality and performance, especially with respect to their technology-enhanced items. They should be withdrawn from the market before they precipitate a national catastrophe.”
[Brandenburg:] Here is some of the rest of his critique:
“Flaws in the Smarter Balanced Test Items
“What happened? Despite elaborate evidence-centered design frameworks touted by Smarter Balanced as our assurance that their tests would measure up, the implementation of the tests is egregiously flawed. I wish I could say the flaws in the Smarter Balanced tests are isolated.
“Unfortunately, they are not. While the shortcomings are omnipresent and varied, they fall into categories, all illustrated multiple times by the examples in this critique:
• Poorly worded and ambiguous mathematical language and non-mathematical instructions;
• Incorrect and unconventional mathematical graphical representations;
• Inconsistent mathematical representations and user interfaces from problem to problem;
• Shoddy and illogical user interface design, especially with respect to the dynamic aspects of the mathematical representations; • Consistent violations and lack of attention to the Common Core State Standards;
• Failure to take advantage of available technologies in problem design….
“The result? Untold numbers of students and teachers in 17 Smarter Balanced states will be traumatized, stigmatized and unfairly penalized. And the quagmire of poor technological design, poor interaction design, and poor mathematics will hopelessly cloud the insights the tests might have given us into students’ understanding of mathematics.”
Rasmussen then analyzes sample SBAC test questions.
Diane Ravitch's blog 
Right on!
Wasn’t it Bill Gates who said it would take ten years to see if this reformer crap works. That the reason so much with the tests is crap because it could take 10 years to fix it all while gathering the evidence to see if it all works.
But we know they will make it work by playing with their numbers. Did I say playing? :o)
Lloyd, did Bill Gates say when the 10 years started? Like, are we in year 5? Year 8? When can we say, “enough”?
I was ready to say enough back in the 1980s. :o)
Didn’t he say the same thing about Windows, ten years to fix it? And what are we now, year 30 and still no fix.
Gates doesn’t fix things. He patches them.
His products are like an old holey (certainly not holy) pair of jeans with so many patches that you can’t even see the original material.
Poet writes, in prose, “Gates doesn’t fix things. He patches them.”
This is the issue: there’s no perfect software, while online tests with their built in assessment systems need to be perfect without any bugs.
In contrast, if a test is designed, administered and graded by the kids teacher has some imperfection, the kids can give immediate feedback and the teacher can offer immediate corrections.
Whenever I sit down to help my kids with online home work or practice tests, I feel claustrophobic: I feel confined in the interface, the method of assessment, the choice of problems, the choice of answers.
My kids, all their friends and the teachers I talk to feel the same.
“there’s no perfect software”
True enough, but some software is far better than other software. Some software actually goes through an extensive specification/design/development and testing process before it is ever released (I know because I worked for a company that is probably one of the best in that regard)
Gates doesn’t even know what good software (or anything else) is.
In fact, he doesn’t care because, as you rightly point out, he just figures he can release something that kinda works and let the users find the bugs (which he can then patch up with the patch of the day).
Bill Gates should never have been let anywhere near a process to develop standards because he has absolutely no clue what a high standard (good quality) which translates to good quality) is.
Microsoft has low standards and low quality.
Even worse is Internet Explorer. They are scrapping that flop. Same view towards teachers – if those pesky humans are too buggy, just discontinue and make a different one.
I know all about Bill’s patches that Microsoft calls updates.
They arrive about once a week (or more—just checked and discovered that new updates are waiting, and I spent a few hours updating last Saturday) and disrupt my work because I have to Restart the computer after every update and that means I have to save and shut down everything I’m working on, because if I don’t, when I turn the computer back on, my work is not in the same order as before.
I’ve also discovered that if I wait too long to update, my system starts to rebel and have problems—until I do the update.
Loyd Lofthouse writes “That the reason so much with the tests is crap because it could take 10 years to fix it all while gathering the evidence to see if it all works.”
Well, this is the thing: these tests are software experiments, and the companies , through politicians, mandate the whole country to take part in the debugging process.
Normal experiments conducted by normal people are done in labs, and even there only consenting humans are allowed to be used.
Gates has no problems having billions of people test his buggy products while making billions. That’s his actual business model.
It’s another matter that without online tests (as well as Windows), the World would be a better place.
It also seems that Bill Gates has no problem punishing everyone who doesn’t update their endless Microsoft patches on a regular basis—like weekly. When I let the updates wait longer than one week and keep working, the computer eventually starts to have problems and soon I can’t work unless I stop everything and update.
I’ve also noticed that when I shut down Explorer—I use Google Chrome, Firefox more and only use Explorer depending on a function that the other two don’t offer—Explorer is the only one where that little blue arrow chases its tail for a long time after I log out.
I have often wondered if that means Microsoft Explorer is gathering data from my system about me to sell to someone else or playing cooky spies on my system to continue to gather data on me to sell to someone else. I don’t see the same thing happening with Google Chrome or Firefox when I log out of those two.
For that reason, when I log out of Explorer, I quickly delete all of the cookies and the history of where I have been hoping to foil Bill’s data collection for sale greed.
Reblogged this on Exceptional Delaware.
The major flaw of standardization. When something is a flop like CCSS, SBA, and PARCC, the negative impact is propagated everywhere and amplified. The momentum perpetuates the argument that we can not stop doing the wrong thing just because it is wrong. The fact innovation and individualism are stifled means there are no self correcting mechanisms to prevent disaster. Eventually, whether Wall Street, USSR, or New Coke, the system comes apart and collapses. People like Gates, Walker, and Duncan will walk away unscathed. The teachers will be left to clean up the mess.
My little rural school in Nevada was informed that our computer capacity can not accommodate the SBAC. We are unable to load a special secure browser, we can not use Windows XP to run it….The technical specialists for the Clark County School District are working on it but they do not think they can do what is needed before the testing window closes. They have considered adding a week to the school year if they can get the test to run. I already told them not to call me, I will not come back to help administer this abomination. I will take medical days and have some health checks if necessary.
Isn’t there a paper/pencil version available? We just took the parcc that way (also at a very small rural school) and I can tell you, as the tech teacher/director, I was thrilled. The amount of email going around, our tech listserv, with parcc related problems was in the thousands per day! OK it only seemed like thousands but I was overwhelmed with the amount of problems. I can’t imagine doing this test online next year. There are literally hundreds of hoops the tech person has to jump through just to get the kids to the test, not to mention all the tweaking and manipulating the system has to go though to keep them online during the test. There is also a host of exit procedures the tech are still going though!!!! ahhhhhhhh take those days off if you can!
Our district does not have a single paper copy available. Just like any reckless gambler, Nevada is all in on the high tech version. It should crash magnificently. We have another bloated technological marvel from Tennessee called infinite campus. Our computers barely crawl from the overload they work under. A crash is inevitable according to the tech wizards. Their smoke and mirrors can’t fix this problem.
I agree. Our schools are teaching students how not to think.
“They should be withdrawn from the market before they precipitate a national catastrophe.”
But that’s the whole point.
It is a national catastrophe.
Maybe Coleman didn’t realize just how right on he was when he stated he was unqualified – not only in ELA but more so in Math. Seen here, for example: http://truthinamericaneducation.com/common-core-state-standards/david-coleman-2-years-ago-we-were-a-collection-of-unqualified-people/
Yet if you want to close the Gap – confuse the text takers… Are they not designed to fail 6 and 7 in 10? They (the standards included) are not designed to lift up, but rather drag down. EVERYONE is in agreement on issues with the Math.
Coleman states he was unqualified.
Zimba states the standards only prepare for 2 yr college- and NOT for STEM
Packard says AP Calc outside of the standards — drop it and offer AP Stat
Psychologists all over state they are developmentally inappropriate
The two content experts refused to sign off.
Missing Delayed or Muddled Topics (Milgrim/Wurman) http://www.doa.nc.gov/asrc/documents/03162015/ErrorsInCCMathStandardsMilgram-Wurman.pdf
Common Core a Floor https://twitter.com/educationgadfly/status/573206269760307200
Can Gifted Students Survive: http://edexcellence.net/articles/can-gifted-education-survive-the-common-core
But then the 21st century economy requires it –?
I watched the video linked to from the article. Interestingly, I had similar problems with taking an online PARCC test for 3rd graders that was posted on the Badass Teachers’ facebook page. Unlike the kids, who do not have this option, I gave up after not being able to enter the denominator for a fraction.
I am a math prof and because of my my two kids, I have had to suffer through hundreds of online homework assignments, sample tests.
I think the article is a bit misguided in its focus: it’s criticizing the technical implementation of the tests, the design of the problems on the tests, and it sort of implies that one just has to improve the technology, and be more careful about designing the problems. This is a mistake.
Let’s not give an excuse to the test maker industry to continue their work. Their work is useless in education.
I think the very reason that many (maybe most) teachers are sold on the idea of these online tests, homework assignments is that they are supposed to make assessments easier, automatic . Teachers do embrace them, because they have to do an insane amount of grading and computers give them some relief.
But these tests don’t test real math understanding. No technology (and no written test) can replace the individual assessment of a student’s understanding of math by the teacher.
The real solution is to make sure a teacher doesn’t have more than 15 students, and doesn’t teach more than 4 classes a day. Then she will be happy to do what made her to become a teacher in the first place: make students excited about math, and to be able to pick their minds individually.
Then we can forget about centrally administered tests and insane amount of homework.
Various compromises that are made with testing companies and politicians just allows the dark side to make more and more steps in taking over the teaching profession and our kids education.
“Consistent violations and lack of attention to the Common Core State Standards;”
That’s probably actually a plus.
An interesting email….. I really need to work on my GROWTH mindset here…
Dear Staff,
Some of you have expressed concern about the administration of the Smarter Balanced Assessment (SBA). Concerns have included questions about the utility of the assessment, the instructional time lost due to testing and the likelihood of lower test scores. I encourage you to visit our newly developed Frequently Asked Questions on our district website.
I would acknowledge that the SBA presents unique challenges – particularly in regard to this year’s test administration. Although the amount of SBA testing time for each individual student is relatively small (about eight hours depending on grade level), because of our limited technology, the administration of the test is spread out through much of the spring months.
The SBA however, does take several steps in the right direction. It reflects the higher College and Career Readiness Standards (Common Core) which our students need to compete successfully in a global market for family wage jobs (75% of family wage jobs now require SOME college). Yes, the SBA does measure student progress against a higher standard and the percentage of students that achieve proficiency will likely be lower. We have seen similar trends with the WASL, the MSP, the HSPE, and the EoCs. In each case, we have learned from the test results and risen to the challenge. I am confident that we will do so again, and that the preparation work done in Washington and Seattle will mean that our state and district scores will continue to compare favorably. Seattle currently outperforms the state in Reading, Math, Science and Writing.
I also take seriously my oath of office promising to uphold state and federal laws. The SBA is a state requirement and a part of both state and federal law. Since we are already a high risk grantee for Special Education, failure to give the SBA could further jeopardize our federal funds. Teacher and principal contracts, as well as state certification, also require adherence to state law.
All employees responsible for administering the SBA are directed to give the SBA as required by law and/or their supervisor. I would also ask that staff give appropriate professional advice (rather than personal advice) to parents.
SBA administration has begun for many of our schools and will begin soon for many more. I want to thank all of those who have worked so hard to prepare for these assessments – from student learning, to keyboarding, to technology, to the logistics of scheduling, and more.
A note to those with personal/professional objections to administering the SBA:
Staff who object to administering the SBA must give advanced notice to their supervisor and work with their supervisor to insure that arrangements are made for SBA administration and coverage. For those who give advanced notice, refusal to give the SBA will be considered as misconduct. Consequences will be determined after further consideration and review of the CBA. Failure to give advanced notice of your refusal to administer the SBA, will be considered as insubordination and flagrant misconduct, and it will result in the imposition of serious discipline, up to and including termination of your employment and a referral to OSPI to take action on your certificate.
Finally, I want to restate my confidence in the teachers and students of Seattle. We live in challenging times. Each of us – superintendent, principal, teachers, and students – faces increased challenges in support of student learning. There is no question, we are each being asked to do what seems daunting. The challenge of our times is whether we face those challenges with a GROWTH mindset or a FIXED mindset. A fixed mindset says we are what we are and have no hope for future growth. A growth mindset says that we can each learn and grow through dedicated effort. Seattle has demonstrated that kind of growth mindset resulting in a steady increase in test scores. The growth over the past six years has been exemplary district-wide and includes more schools of distinction than any other district in Washington. We have that hope and aspiration for each of our students and each of us as colleagues and professionals.
Thank you – each of you – for your part in making great things happen for each and every one of our students.
Sincerely,
Larry Nyland, Superintendent
PS: Although the focus of this memo is Smarter Balanced Assessments, I recognize the challenges presented by the combination of SBA, district interim assessments and other building assessments like the SAT and WAKids. I have had initial talks with SEA and look forward to further conversations about creative options that will help us focus more clearly on the assessments that matter the most for learning.
Sorrell writes “The challenge of our times is whether we face those challenges with a GROWTH mindset or a FIXED mindset. […] Seattle has demonstrated that kind of growth mindset resulting in a steady increase in test scores. ”
Growth can mean all kinds of development in all kinds of directions. I think many here argue that growth in test scores don’t reflect student learning and teachers’ quality.
But the main focus of the post was a very detailed criticism of a particular test, SBA, while your response lacks details and seems off the track about laws and administering the tests. So what was the purpose of the response?
Sorry for being unclear. I was sharing an email from the superintendent that was really disconcerting to me. I didn’t write it myself.
On second reading, you were clear. I just forgot your intro sentence by the time I got to the end of the response. Sorry.