Jeffrey Weiss here writes in the Dallas Morning News about “How the Texas Testing Bubble Popped.”
One man, the courageous State Commisssioner of Education, Robert Scott, said what was on everyone’s minds.
Everyone thought he was a loyal soldier in Governor Rick Perry’s army, a slave to standardized testing.
But then he said the words that gave hearts to parents and educators in Texas and across the nation.
Diane Ravitch's blog 
Robert Scott did the right thing and it cost him his position as Education Commissioner. The grassroots movement that changed the secondary testing requirements from 15 to only 5 instilled a little logic into the system. Now if this same type of logic could be applied to the elementary and junior high level testing, and if the testing would be written on a developmentally appropriate level for students and if the test were to be used for diagnostic purposes, Texas might be the first to “do it right.” I fear; however, that Pearson has its spurs in deep and this is going to be a long uphill battle.
I agree.
“Texas might be the first to ‘do it right.'”
There is no “doing it right” when it comes to educational standards and standardized testing. Wilson has proven the complete invalidity of the whole process. Start with invalid processes end with invalidities. I can’t be gotten right. To understand why download and read Wilson’s “Educational Standards and the Problem of Error” found at: http://epaa.asu.edu/ojs/article/view/577/700
Brief outline of Wilson’s “Educational Standards and the Problem of Error” and some comments of mine. (updated 6/24/13 per Wilson email)
1. A quality cannot be quantified. Quantity is a sub-category of quality. It is illogical to judge/assess a whole category by only a part (sub-category) of the whole. The assessment is, by definition, lacking in the sense that “assessments are always of multidimensional qualities. To quantify them as one dimensional quantities (numbers or grades) is to perpetuate a fundamental logical error” (per Wilson). The teaching and learning process falls in the logical realm of aesthetics/qualities of human interactions. In attempting to quantify educational standards and standardized testing we are lacking much information about said interactions.
2. A major epistemological mistake is that we attach, with great importance, the “score” of the student, not only onto the student but also, by extension, the teacher, school and district. Any description of a testing event is only a description of an interaction, that of the student and the testing device at a given time and place. The only correct logical thing that we can attempt to do is to describe that interaction (how accurately or not is a whole other story). That description cannot, by logical thought, be “assigned/attached” to the student as it cannot be a description of the student but the interaction. And this error is probably one of the most egregious “errors” that occur with standardized testing (and even the “grading” of students by a teacher).
3. Wilson identifies four “frames of reference” each with distinct assumptions (epistemological basis) about the assessment process from which the “assessor” views the interactions of the teaching and learning process: the Judge (think college professor who “knows” the students capabilities and grades them accordingly), the General Frame-think standardized testing that claims to have a “scientific” basis, the Specific Frame-think of learning by objective like computer based learning, getting a correct answer before moving on to the next screen, and the Responsive Frame-think of an apprenticeship in a trade or a medical residency program where the learner interacts with the “teacher” with constant feedback. Each category has its own sources of error and more error in the process is caused when the assessor confuses and conflates the categories.
4. Wilson elucidates the notion of “error”: “Error is predicated on a notion of perfection; to allocate error is to imply what is without error; to know error it is necessary to determine what is true. And what is true is determined by what we define as true, theoretically by the assumptions of our epistemology, practically by the events and non-events, the discourses and silences, the world of surfaces and their interactions and interpretations; in short, the practices that permeate the field. . . Error is the uncertainty dimension of the statement; error is the band within which chaos reigns, in which anything can happen. Error comprises all of those eventful circumstances which make the assessment statement less than perfectly precise, the measure less than perfectly accurate, the rank order less than perfectly stable, the standard and its measurement less than absolute, and the communication of its truth less than impeccable.”
In other word all the logical errors involved in the process render any conclusions invalid.
5. The test makers/psychometricians, through all sorts of mathematical machinations attempt to “prove” that these tests (based on standards) are valid-errorless or supposedly at least with minimal error [they aren’t]. Wilson turns the concept of validity on its head and focuses on just how invalid the machinations and the test and results are. He is an advocate for the test taker not the test maker. In doing so he identifies thirteen sources of “error”, any one of which renders the test making/giving/disseminating of results invalid. As a basic logical premise is that once something is shown to be invalid it is just that, invalid, and no amount of “fudging” by the psychometricians/test makers can alleviate that invalidity.
6. Having shown the invalidity, and therefore the unreliability, of the whole process Wilson concludes, rightly so, that any result/information gleaned from the process is “vain and illusory”. In other words start with an invalidity, end with an invalidity (except by sheer chance every once in a while, like a blind and anosmic squirrel who finds the occasional acorn, a result may be “true”) or to put in more mundane terms crap in-crap out.
7. And so what does this all mean? I’ll let Wilson have the second to last word: “So what does a test measure in our world? It measures what the person with the power to pay for the test says it measures. And the person who sets the test will name the test what the person who pays for the test wants the test to be named.”
In other words it measures “’something’ and we can specify some of the ‘errors’ in that ‘something’ but still don’t know [precisely] what the ‘something’ is.” The whole process harms many students as the social rewards for some are not available to others who “don’t make the grade (sic)” Should American public education have the function of sorting and separating students so that some may receive greater benefits than others, especially considering that the sorting and separating devices, educational standards and standardized testing, are so flawed not only in concept but in execution?
My answer is NO!!!!!
One final note with Wilson channeling Foucault and his concept of subjectivization:
“So the mark [grade/test score] becomes part of the story about yourself and with sufficient repetitions becomes true: true because those who know, those in authority, say it is true; true because the society in which you live legitimates this authority; true because your cultural habitus makes it difficult for you to perceive, conceive and integrate those aspects of your experience that contradict the story; true because in acting out your story, which now includes the mark and its meaning, the social truth that created it is confirmed; true because if your mark is high you are consistently rewarded, so that your voice becomes a voice of authority in the power-knowledge discourses that reproduce the structure that helped to produce you; true because if your mark is low your voice becomes muted and confirms your lower position in the social hierarchy; true finally because that success or failure confirms that mark that implicitly predicted the now self-evident consequences. And so the circle is complete.”
In other words students “internalize” what those “marks” (grades/test scores) mean, and since the vast majority of the students have not developed the mental skills to counteract what the “authorities” say, they accept as “natural and normal” that “story/description” of them. Although paradoxical in a sense, the “I’m an “A” student” is almost as harmful as “I’m an ‘F’ student” in hindering students becoming independent, critical and free thinkers. And having independent, critical and free thinkers is a threat to the current socio-economic structure of society.
As we wait for these people to have their epiphany, the damage continues. Here in Ohio, we are still scratching our heads over the House Republicans rejecting a Senate bill modifying the over reliance on test scores. The senate bill passed unanimously in a Republican controlled government. Yet the House demanded provisions to fire all teachers scoring “ineffective” on the tests, make it harder for veteran teachers to achieve good evaluations, and demand student surveys or 50% of an evaluation depend on tests. These people think “teacher” is a four letter word.
MathVale, from my understanding of the House edition of the SB 229, they never deviate from 50% MINIMUM of teacher evaluation of test scores. Other highlights include Ohio Department of Ed (ODE) come up with standardized measurements for every class of every K-12 student by 2016, teachers with “ineffective” rating (the lowest of 5 categories) in consecutive years are not allowed to teach in Ohio, and teachers who have been teaching for at least 10 years are not allowed to have a “developing” rating (2nd lowest category) more than once. The new rating grid also makes it impossible for a teacher whose standardized test scores are in the lowest category to be higher than “developing” rating, regardless of administrative evaluations.
“union” must have four letters. too?
Parents must opt-out of tests and data mining. At the same time, taxpayers must demand the full release of all high-stakes tests and corporate testing contracts.
Time for the same nationwide. Only this time, we need to be more thorough. No more Son of NCLB, no more NCLB Fright Night II: the Nightmare Is Nationalized.
Time to put a stake in standardized testing once and for all.
“Time to put a stake in standardized testing once and for all.”
No doubt. And that stake was forged by one Noel Wilson. We need to use it! (see above for reference)
I love the Wilson piece. He’s a bit wacky, but he is asking precisely the right questions. Education desperately needs its Wittgenstein–the one who will question (Derrida would have said “interrogate”) the foundational concepts. What do we mean by “measure,” here? What is measurable and how? What do we mean by “standard”? What will be the consequences of all this?
Wittgenstein repeatedly urged people not to fall back on their preconceived notions but actually “to take a look,” to “look carefully.” Doing so isn’t easy because it’s difficult for people to see what is so familiar to them, to push past all the unconscious learning that they have done, their preconceptions, prejudices, interpellations. This phrase is often attributed to Einstein: “Common sense is that layer of prejudices laid down before the age of eighteen.” Well, people have a lot of “common sense” about measurement that turns out to be flat out wrong, and Wilson has some good instincts about what’s wrong. But he writes sloppily and overstates things and opens himself to the kind of critique that TE offers of him. I am not criticizing, here. What Wilson is doing is VERY important, and it’s NOT EASY to explore these preconceptions, to figure out how to begin to see the familiar as, in fact, the strange.
As a Texan, be carefule what you ask for. The same people who reduced secondary school tests from 15 to 5 also eliminated Algebra II as a graduation requirement. They also want to greatly increased charter school, cut the budgets, teach creationism, and every bad educational idea. The anti-testing movement is comprised of strange bedfellows of left and right.
You are speaking of the disfunctional Texas State Board of Education – Not TAMSA.