Gene V. Glass, one of our nation’s most distinguished researchers, ponders whether the opt out movement can succeed in curbing the abuses of overuse of standardized testing.
He says that it would not take massive percentages of students opting out to disrupt the system:
At present, the Opt Out movement is small — a few thousand students in Colorado, several hundred in New Mexico, and smatterings of ad hoc parent groups in the East. Some might view these small numbers as no threat to the accountability assessment industry. But the threat is more serious than it appears. Politicians and others want to rank schools and school districts according to their test score averages. Or they want to compare teachers according to their test score gains (Value Added Measurement) and pressure the low scorers or worse. It only takes a modest amount of Opting Out to thwart these uses of the test data. If 10% of the parents at the school say “No” to the standardized test, how do the statisticians adjust or correct for those missing data? Which 10% opted out? The highest scorers? The lowest? A scattering of high and low scorers? And would any statistical sleight of hand to correct for “missing data” stand up in court against a teacher who was fired or a school that was taken over by the state for a “turn around”? I don’t think so.
Now THERE is a valuable use of Clayton Christiansen’s theory of disruption as a positive force for change!
Diane Ravitch's blog 
When are teachers’ unions going to refuse to negotiate evaluation plans that include student test scores on these and other inappropriate, unnecessary and irrelevant tests? I think teachers are “abused” just as much as students. I wish teachers could opt out or refuse to give these tests. Maybe more teachers will opt out even though they may lose their jobs. They’re being “rated” on an unfair scale anyway.
LikeLike
So now I know where the less than 95% participation consequence came from. In Ohio, if a school or district has less than 95% of its student population taking the high stakes tests, they are penalized ten points on their state report cards. This penalty would drop their score an entire letter grade.
LikeLike
The 95% minimum test participation requirement was part of the original NCLB act.
It was intended to limit schools from culling their poorest performers from testing. Under NCLB. the AYP requirement was applied to 10 different subgroups. With sub-group minimums at 30. eliminating even a few key students could have avoided federal penalties.
LikeLike
I am aware of that, NY teacher, but that doesn’t change my initial comment. Families opting out have nothing to do with schools hiding low performing students, and yet, schools are penalized. And the reason schools would be hiding those students is that they are trying to avoid the punitive actions that result from low test scores.
LikeLike
Another the way the opt out movement can succeed is by giving support to those teachers and administrators who are criticizing excessive testing. The superintendent in my district now has a blog post saying that our excessive testing is “insane.” His word. He is very concerned about excessive testing reducing instructional time. He is working on a plan to reduce the time spent testing and prepping for tests. This is movement in the right direction. He can refer to the opt out movement as providing evidence that parents think that too much time is wasted on tests.
LikeLike
Test items being posted on social media represent an even bigger threat to so called “test validity” because there is absolutely no way of knowing how many (and which) students have seen the information.
Pearson clearly sees that as the biggest threat, given the lengths they are willing to go to to spy on kids.
Then again, if the self-called statisticians at places like Pearson actually knew or cared about “validity”, they would not be working where they are.
The whole standardized test/VAM scheme is a fraud and no legitimate statistician would ever take part in it.
LikeLike
AMEN.
LikeLike
You are quite right poet. In fact, the American Statistical Association has roundly blasted VAM and stated that flipping a coin would be more accurate and valid.
LikeLike
I am not a statistician, but I looked at the issue of missing data in Ohio where the contract for tests is with SAS and the way it deals with missing data is described. The problem of missing data is treated in two ways. When the scores are missing for students, the system just processes all available scores with elaborate checks for outliers, double entries, and the like. Every available score that can be linked to a teacher of record is used.
The plot thickens in evaluating teachers and schools. In that case the system will impute a missing score for a student –make one up– following some set of rules. Gene Glass suggests some of the rules and decisions that can be made.
In Ohio, SAS puts in an invented mean score for a missing score in order to run the VAM. SAS calculates a mean score for the student on prior tests, and looks at prior performance of the students taught by the teacher.
I am not a statistician but the EVASS system in Ohio is vigorously defended and with no small degree of arrogance in technical reports, most of these a legacy from Dr. William Sanders, and his students, who had applied the VAM methodology in studies of generic engineering–the value-added to the output of seeds, sows, and cows. Dr. Sanders viewed teachers as producers of “value added” to the test scores of students. His first studies used test scores made available to him by officials in Tennessee.
So, teachers have productivity measures, VAM, comparable to those of generically modified seeds, sows, and cows.
Elsewhere I have discussed how some of the hazards in genetic engineering are also applicable to the condition of public education, and values being promoted by current policies, including for example, the principle of survival of the fittest, and large scale experiments with “interventions” analogous to studies of crop yields (say, for college and careers, or reading with fluency by grade three).
LikeLike
Economists are notorious for misapplying mathematical models that were designed for something else. It’s what they do “best”. 🙂
LikeLike
In the early days of man, survival of the fittest had to do with staying alive when faced with one’s enemies, whether that be the environment, other species, or fellow man. One would think humans have progressed beyond narcissism and some level of altruism would be evident. But the 1% evidently feel they are the fittest and that their survival depends upon destroying the less deserving. Scary stuff when you read about the connections between prominent humans and Nazi Germany.
LikeLike
As the politicians take no heed of statistical evidence now, one wonders if even this would get their attention.
An ingrained set philosophy not based on factual evidence is extremely difficult to over come and obviously so far the statistics seem not to have any effect, especially when money is to be made. One would hope that it would and perhaps it would. It certainly would not hurt to try. All weapons at our disposal should be used if they prove effective and maybe in conjunction with other “weapons” might finally get the job done, hopefully before the damage is so extensive that it may be impossible to rectify it.
LikeLike
The movement could succeed but we have taught, most, of our students to be good and follow rules. They don’t like testing, for the most part, but they hate letting the adults around them down even more. They are the ones who have to say ‘I refuse’ and that is a hard thing for a kid to say (in IL). I wish they could just know it’s OK and it counts for NOTHING…just sit back are read a book for a few days, no one will be disappointed in you.
LikeLike
There is no tipping point – other than 100% refusals. Whatever scores they get, they will use. When harm occurs through the use of partial scores, litigation may decide the less minimum participation that can be used to evaluate teachers, programs, or schools.
Arne could care less. He recently said, “Millions of kids around the country are taking the tests. We’re fine.”
LikeLike
NY teacher, so are you saying that it’s pointless for a parent to have their child refuse?
LikeLike