Tennessee is still Racing to the Top although they are still far away. So, of course, the state switched to online assessment for its Common Core testing, at a cost of $108 million.
Yesterday was the first day, and the system crashed.
There was a “major outage.” The state commissioner, a huge fan of Common Core, blamed the vendor. She told schools to go back to the “worst case scenario,” that is, pencil and paper testing.
Since we learned not long ago that students who took the PARCC tests on paper instead of on a computer tend to get higher scores, this may have a bright side.
Diane Ravitch's blog 
The major downside, though, is the amount of time and money spent on preparing for the online test. Major investments were made and faith was placed in a small-time vendor when we could have plugged into PARCC instead.
My understanding is that the “small time vendor” subcontracted with AIR.
Or you could have saved all that time and money by not doing BS testing at all.
when will they ever learn, when will they ever learn…
My daughter took these MIST tests a few times, and her experience (along with other students) should have halted online testing immediately.
She told me this morning that reading a text in the English MIST required scrolling, so you couldn’t see the whole text in one piece, which is an enormous problem since you were supposed to answer questions based on the text. But the scrolling up and down was extremely slow. Then, it was impossible to enter “free” answers because pressing a key on the keyboard often didn’t result in anything appearing on the screen.
The school (or school district) apparently spent a lot of money on small laptops on which some of these tests were administered. The small screen made the tests worse since more scrolling was needed.
Let’s face it, no online testing will ever work which relies on computers’ remote connectivity—even if remote only means a server inside the school.
The only reason for online testing is to enable mass testing. Actually, the main reason for written tests was probably the same: more standardized, more uniform tests and evaluations.
Online and written tests are sold as “fair” to all. Well, I think they are uniformly bad for all.
It’s time to go back to live conversations between teacher and student—not exactly a revolutionary thought, is it?
There are some things that a paper and pencil test do better, and reading comprehension is one of them. Good readers go back to text to reread sections, search for information, draw conclusions and correct misunderstandings. In this case technology is a hindrance to the process. That may explain why students that took pencil and paper tests scored higher than they did on a cyber version.
MIST is not a bad acrynym for another version of expensive vaporware.
The scrolling issue is perhaps one one of the main reasons why kids seem to be doing better on paper and pencil tests that tests in this format. Of this was a “sort of local” or regional subcontracor, and the platform was really from AIR, that would be good to know.
Well, Laura, here’s a quote
“2. Measurement Inc. is subcontracting to AIR, a much larger player in the country’s testing market. AIR already has contracts with Utah and Florida, so Tennessee educators will be able to compare scores of Tennessee students with students from those states “with certainty and immediately.” AIR is also working with Smarter Balanced, one of two federally funded consortia charged with developing Common Core-aligned exams. That means that educators in Tennessee will also likely be able to measure their students’ progress with students in the 16 states in the Smarter Balanced Consortium.”
from http://tn.chalkbeat.org/2014/11/12/measurement-inc-inks-108-million-testing-contract/#.VruUTiAXDMU
I teach in Tennessee. Given the problems we faced with MIST simply trying to access the practice questions, it’s not surprising the online platform was a debacle on test day.
Are they shipping this stuff out without testing it? Are the kids the testers? Because that’s what it looks like.
Why do we consistently give kids such garbage tools when we’re demanding they do these difficult jobs?
With all the adults strutting around crowing about how kids need “rigor!” and “higher standards!” one would think they could see fit to give them a tool that works.
I just found out this morning that my daughter has been taking these MIST tests for a while. So I guess they have been testing it with kids for a while. And they did have problems but they ignored them. Similarly to the US DOE according the CIO hearing video https://dianeravitch.net/2016/02/07/you-must-sit-down-and-watch-this-video/.
I think, they have been getting away with crap for too long.
I mean, we have this blog exactly because these people in power are deaf to teachers’, parents’, students’ complains.
Chiara,
There are HUGE costs in testing a whole system like MIST, so the contractors shortcut field-testing with real students. Tests are usually limited to the overall platform—the capacity of the servers and delivery system, with schools reporting problems as discovered. That trouble shooting process appears to have failed miserably in Tennessee.
Testing with actual students and schools —authentic field tests—is avoided for another reason: A large number of students would be exposed to test items. Each item is worth a lot of money, one reason why test security is a big deal to testing companies.
Then there are “practice effects” with any field test, and these make the actual tests invalid. Once-upon-a time, all test prep was a taboo, looked upon as a cheating. But that is rarely talked about anymore. It is off the table because so many people are trapped into puffery about the importance of test scores, especially the supposed efficiencies of online tests, delivered with metrics and graphs and color coded dashboard reports that look really, really, truly important.
Apart from those matters, students are almost always un-reimbursed field testers for at least some of the items on the actual tests. And parents are not notified so they can give or withhold permission for this “guinea-pig” factor. These extra items are not counted in the score the student receives, but the test maker analyzes the scores later, a general process dubbed “item analysis.” The item analysis determines if a particular item is a decent candidate to go into an “item pool” for later use. The analysis shows the proportion of students who answered correctly or appropriately. That analysis indicates the relative ease/difficulty for the grade-level and so on.
So for the opt-out movement gurus, an additional issue about the testing industry (depending on the test) is whether parents have been notified of any “guinea-pig” factor, unreimbursed child labor, and potential violations of parental rights bearing on full disclosure. Just saying, not a lawyer.
Several years ago in Florida, about 10% of the items on paper and pencil tests were there only for field-testing. They were not counted toward the student’s score. That information was disclosed in a series of papers available at the state department of education website, but the primary audience was not the really public. The papers were for other workers in test development.
Companies were putting tests into the marketplace with the help of a lot of “free labor” from students and their teachers.
AS others have noted, the test in Tennessee appears to have been subcontracted to AIR in order to expand the opportunity to compare scores across multiple states where AIR has contracts for comparable tests—Utah, Florida, and at least some of the 16 states where AIR is likely to have secured contracts for Common Core-aligned exams.
Tennessee may have has an incentive to go for this low-bid contract for another reason: Jobs.
Measurement Inc., and has had a scoring center in Nashville for a long time. Source: http://tn.chalkbeat.org/2014/11/12/measurement-inc-inks-108-million-testing-contract/#.VruQ6kJqvzI
LAUSD. John Deasy. Train wreck iPads and catastrophe MISIS = a third of a billion dollars. [And that’s just the big ticket items!]
TENN. RTTT. CCSS online testing. $108 million.
When it comes to squandering scarce resources, John Deasy & Rheephorm Inc. could still teach Tennessee rheephormsters a thing or two about starving public schools into submission.
😏
My small brain says ” Why not give the questions on paper and use the computer for the answers?”
Since it’s difficult or impossible to enter answers into the computer?
My brain questions “Why the Sam Hell are we testing students using COMPLETELY INVALID standardized tests?”
Here, gimme that 16 lb (computer tests) hammer to replace the 12 lb (pencil and paper tests) hammer so that I may beat my brains into mush faster.
“. . . this may have a bright side.”
Yeah kind of like surviving a plane crash, waking up in the hospital and being alive but paralyzed from the neck down with a major loss of cognitive abilities, not knowing who you are and no speech capabilities.
The interesting thing about the failure of the Tennessee MIST experiment is that the massive failure seemed to come out of the blue if you read the press releases. We had the same problems when we took class time to have a trial run back before Christmas. Why were officials surprised?
I have yet to hear anyone talking about throwing out the test results altogether due to the switch back and forth between modes of delivery. I bet they still count as ten percent of our evaluation. It’s not enough that the standards are changed every few years, now we have to change the mode of testing too. For us teachers, it is like throwing darts at a moving board. We have been spending time all year to get the kids ready for the actual mode of testing. Time was not entirely wasted, for some of the questions were good ones, and it is always interesting to do something different. Still, trying to achieve a coherent structure to class was made more difficult by this whole process. The least they could do would be to admit that this test has no bearing on teaching performance, and take away that aspect of teacher evaluation until things get settled.
I add that I was told that MIST counts for 10% for a student’s grade as well. This failure of the online system would be a perfect reason to opt out (excuse me for bad language, “refuse”), but what if my daughter gets a worse grade as a result?
I was told that the test was not counting for the student’s grade. Initially I understood that the students were not to be affected for the first few years. This year teacher evaluation included 10% VAM score based on the TNready. I think the plan was to increase the percent to 35% over a few years which is where we were with our previous evaluation system. Who knows now? Of course, I am terribly out of the loop on specifics, and they have been changing faster than the mind of a politician.
MIST accounting for 10% of a student’d grade seems to be a fact, confirmed by several teachers.
The rumor is that there will be no testing component to teacher evaluation this year.
I teach in TN on the semester block. English III, which is notoriously and historically one of the most difficult tests the state gives already. Now half my kids will have scores from tests taken online and half will have scores from paper. You better bet I’ll be breaking down that data in depth.