A reader of the blog posted the following comment. She asks the question: What can a standardized test tell the teacher that the teacher doesn’t know already? The answer: nothing. To be precise, “absolutely nothing.”
She writes:
One of the most demoralizing moments of my teaching career was being forced to do ACT prep with my secondary ESL students. We would read the questions together, trying to figure out some way of breaking it down into something manageable, and then the students would furrow their brows or just check out completely, and we would all end up frustrated. And I would think, “I’ve spent the last 7 months building a safe classroom community in which students can grow and learn and express their ideas… and then I betray all of that with this absurdity?”
Another demoralizing moment was having to administer the ACCESS test to ELLs. We had to test every single student in the bilingual program even if they weren’t actually taking bilingual or ESL classes anymore. In addition to losing class time, the bilingual department teachers gave up every prep period and lunch period for about 5 weeks to test students individually on oral proficiency. You want to know about the life cycle of the boll weevil? I could tell you. That was on the test two years in a row. One girl had literally arrived to the U.S. the weekend before and enrolled the day before the testing began, and she had to take it. She opened the booklet, flipped through the entire thing not able to answer any of the questions and looked up at me in complete bewilderment. Luckily, I speak her native language and give her some reassurance, but I felt like a failure as a teacher and an abomination of a human being.
Those experiences affected me deeply, and I wish that I had had more knowledge then. I was young and new as a teacher, and frankly, I was overwhelmed. Now, I do my best to read up on what’s going on in education (thank you, Diane, for making that task infinitely easier!), and my mantra nowadays is “opt out.” There is nothing – absolutely nothing – that those tests could tell anyone that I, as the students’ teacher, couldn’t tell them first.
Diane Ravitch's blog 
Thank you, Diane, for featuring my comments here. This is the second time that you have done so. I am humbled and honored by the mere fact that you read what I write and give voice to my experiences.
LikeLike
Laura,
I read every comment on the blog and feature those that I think have wide appeal to readers. Like yours!
LikeLike
It is not whether the tests measure anything of value — you know they don’t and the test pushers know they don’t. What they are pushing is the idea that they should be the ones who assign values to schools, students, and teachers.
LikeLike
“Test Measures”
The test measures wallet
Of those who sell test
Short and the tall of it:
Fattest is best
LikeLike
“. . . the bilingual department teachers gave up every prep period and lunch period for about 5 weeks to test students individually on oral proficiency.”
Pick the correct descriptor of the teachers involved:
A–Fools??
B–Suckers??
C–GAGAers??
D–Bullied??
E–Ignorant??
F–Stupid??
You pick your answer for this standardized test question.
LikeLike
I get what you’re saying, but maybe it’s a “right to work” state.
No union worker should tolerate giving up all preps and lunches for 5 weeks, nor should they. If the admin wants their garbage testing, then they need to schedule time for it.
LikeLike
Not one of the answers, you fail!!!
LikeLike
The ACCESS is a mandatory state test in MA for all ESL coded students. If she refused to administer it, she would have no job.
LikeLike
Tafkattont,
Let’s see Tafkat refers to Kafka and tont is short for tonto which in Spanish is stupid/foolish. Cleverly hidden comment on the state of American public education!!! LOL!
(Hey the caffeine is just starting to kick in on this gorgeous rainy morning here in the beautiful Missouri River Hill Country of Southern Warren County, MO.)
Anyway to your comment. I did not suggest that she refuse to administer it. I am not against using diagnostic standardized tests (no, standardized student achievement (sic) tests are not diagnostic) to help evaluate the needs of the students. What I was hinting at was the insanity of giving up lunch and planning periods to give those tests. Look if the adminimals can’t find subs to cover the test giving teacher’s class during the contracted hours that is their effin problem, not the teachers. Pay the teacher double to do so only if the teacher agrees and the teacher should be able to cancel that agreement at any time. It’s the adminimal’s job to figure out how to get the job done without violating and yes it’s a violation both physically and mentally, the teacher’s rights and being.
I’ve told adminimals in the past that if they wanted me to cover during lunch or planning period they should pay me double on top of being paid for that day. Hey, they can’t find a sub, well that’s their problem, that’s what they get paid the adminimal salary to do find the sub. Tain’t a hard concept to understand but I found that most adminimals couldn’t understand it. They felt someone else should do their job.
LikeLike
Think about the number of economists and statisticians and politicians and investors seeking profits in the education market who depend on these scores. The test scores allow them to make all sorts of claims and unfounded predictions about the fate of children before they enter kindergarten, their fate if they fail to meet grade-level thresholds in test scores, their fate if they belong to subgroups sliced and diced and combined in different ways, their fate if they fail to be college and career ready, the fate of every adult if the kids in school and their teachers fail to save the economy, and so on.
The educational policies of the last 25 years or so have been designed to dehumanize teaching and rid that work of informed professional judgments. In addition to the absurdities captures in this poignant tale, look at the huge amounts of money being poured into the testing industry that should be directed to more humane environments for the work of teachers and the nurture, not intimidation of young hearts and minds.
LikeLike
They are building a commodities market, a currency exchange on top of the education system. The lion’s share of the funds will then go to parasitic brokers who do nothing but speculate on fictional values they themselves set while the peons do all the actual work.
LikeLike
“The Testing Industrial Complex”
The testing industrial complex
Rakes billions in each year
Cares naught for fact or context
And thrives on threats and fear
LikeLike
I’ve been in your shoes with elementary ELLs. I have seen the fear and doubt and helped wipe away more than my share of tears. The inappropriate standardized testing being imposed on ELLs is in direct contradiction to the research on ELLs. Our education system has become so politiicized that the regulations make no sense and are harmful to the very population we are supposed to be helping.
LikeLike
It’s those precious test scores that are the stuff that the marketization of education requires. I still see opt out as the strongest and best tool to fight it.
Opt out. Opt out. Opt out.
LikeLike
The tests are not to tell the parents, teachers or student/test takers anything about how well they are doing. How can they when the tests are 2 grades above grade level, confusing, cumbersome to take on computers, and often have more than 1 correct answer, and feature stories about pineapples and hares?
The tests extract exactly what they are intended to do: Make money for the test maker, make money for the politicians via donations from the test maker via our taxpayer dollars, fail the school, shame/grade the school, fire teachers, close the school, bring in a charter or charter manager. Voila.
LikeLike
While discussing and condemning standards and standardized test scores (yeah, I’m one of the most vocal) we are not seeing the forest for the trees in the next wave of anti-public education malpractices–Competency Based Education (CBE and its many iterations) and it’s sidekick “Invidualized Learning”.
Diane, do you have any posts planned about what is occurring in this sector of edudeformer and privateer malpractice and exploitation? I don’t remember seeing much of anything here about CBE, did I miss some posts? (as poster John found out I don’t always remember everything on this site, my own queries included)
CBE is standards and testing on steroids but it is appearing to fly under the radar for most at the moment.
LikeLike
Duane just put CBE in Diane’s search bar. You will see numerous posts. Here is just one link: https://dianeravitch.net/2016/02/04/warning-digitized-instruction-data-mining/
LikeLike
Thanks. I had looked at the categories at the bottom and didn’t see a category for CBE. Didn’t think of doing what you suggest. I’ll check it out.
LikeLike
And I even responded on that one a couple of times. I’m claiming synthetic opioid dementia-ha ha. (actually the pain med I was on was doing a lot more to me than I realized until I went off of it cold turkey, had 4 days of serious withdrawal-not fun) After getting off the tramadol my attitude, energy level, etc. . . got a lot better and my pain management has become clearer as to what I need to do instead of just blocking the pain.
To all I say beware pain meds such as tramadol which are touted to be non-addicting and harmless. They aren’t-at least for me it wasn’t.
LikeLike
Duane, I know this is OT but I have so much experience/ research w/ this due to my [sadly deceased] eldest w/a long history of both phys & mental-meds history– I could tell you about tramadol et al meds, just email me ginnybee3@comcast.net so sorry you are going thro this
LikeLike
B35,
Sent you an email from my dswacker@centurytel.net account. Thanks for your concern!
Duane
LikeLike
But if we didn’t have standardized test scores, how would the capitalists, CEOs, oligarchs, and pseudoacademics “know” that we were “learning?”
LikeLike
Right as usual, Ed Detective.
None of this is about education nearly as much as it is about shifting wealth and power, and exerting mind control/brainwashing over the increasingly dumbed down masses and expecting them to just go along to get along.
And it’s working! Brilliantly, I must say.
As I go back and forth between Norway and the USA, it is crazy making to say the least. But the powerful plutocrats here don’t scare me nearly as much as the uniformed and misinformed general public.
We are our own worst enemy, but when we get it right and act together, we ward off nearly every form of evil . . .
LikeLike
“Reform” is also about crushing collective bargaining and workers’ rights as well as destroying a democratic institution for “fun and profit.”
LikeLike
The grade 8 NYS science test was held on the first day of Ramadan (Monday, June 1).
I learned that my Muslim students were up at 3 AM finishing their last meal before the sunrise to sunset fasting began – and that hungry, tired, and thirsty (not even water is allowed) students who can’t read English to begin with suffer even more than we could imagine. This was a shameful oversight on the part of NYSED.
LikeLike
Oversight ? or studied indifference?
LikeLike
Testing their grit I suppose. Being tired, scared, confused, hot, hungry, and thirsty is what every newly arrived immigrant should have to overcome to prove that I am an effective science teacher.
LikeLike
Like!
(for Rager’s 7:54 comment)
LikeLike
The ACCESS test is a mandatory test given by WIDA. The testing window is usually the end of January to the first week of March. Unfortunately, you cannot “opt out” of it. Students are tested by grade clusters, which means some students will take the same exact test multiple times (K, 1-2, 3-5, 6-8, 9-12). There used to be a time when parents could refuse ESL services with a letter. Not anymore. Also, there is now a big push to have all ESL students tested online. Unfortunately, there is no tech support. And they expect students to have an annual personal improvement rate of .8. Last year my ESL case manager went through each of my students’ results (all 212) and wanted to know why some students didn’t make progress. I whipped out my trusty Excel sheet that detailed dates, times, room conditions, names of other students that were testing in the same group, student behaviors (did they sleep, talk, seem bored), how long it took each student to complete each section, whether they were present or absent or even moved, if we had a fire drill or lockdown drill or assembly, if I had a coverage and couldn’t test students, etc. My detailed chart was a hit because there was no response from my case manager.
LikeLike
To answer the question of the post:
Nothing! There is nothing to be learned from results that are COMPLETELY INVALID. It’s a total waste of time, energy and resources in order to get NOTHING.
To understand why nothing is the answer read and understand why those results are “vain and illusory” as Noel Wilson says in his never refuted nor rebutted: http://www.edrev.info/essays/v10n5.pdf
“Educational Standards and the Problem of Error” found at: http://epaa.asu.edu/ojs/article/view/577/700
Brief outline of Wilson’s “Educational Standards and the Problem of Error” and some comments of mine.
1. A description of a quality can only be partially quantified. Quantity is almost always a very small aspect of quality. It is illogical to judge/assess a whole category only by a part of the whole. The assessment is, by definition, lacking in the sense that “assessments are always of multidimensional qualities. To quantify them as unidimensional quantities (numbers or grades) is to perpetuate a fundamental logical error” (per Wilson). The teaching and learning process falls in the logical realm of aesthetics/qualities of human interactions. In attempting to quantify educational standards and standardized testing the descriptive information about said interactions is inadequate, insufficient and inferior to the point of invalidity and unacceptability.
2. A major epistemological mistake is that we attach, with great importance, the “score” of the student, not only onto the student but also, by extension, the teacher, school and district. Any description of a testing event is only a description of an interaction, that of the student and the testing device at a given time and place. The only correct logical thing that we can attempt to do is to describe that interaction (how accurately or not is a whole other story). That description cannot, by logical thought, be “assigned/attached” to the student as it cannot be a description of the student but the interaction. And this error is probably one of the most egregious “errors” that occur with standardized testing (and even the “grading” of students by a teacher).
3. Wilson identifies four “frames of reference” each with distinct assumptions (epistemological basis) about the assessment process from which the “assessor” views the interactions of the teaching and learning process: the Judge (think college professor who “knows” the students capabilities and grades them accordingly), the General Frame-think standardized testing that claims to have a “scientific” basis, the Specific Frame-think of learning by objective like computer based learning, getting a correct answer before moving on to the next screen, and the Responsive Frame-think of an apprenticeship in a trade or a medical residency program where the learner interacts with the “teacher” with constant feedback. Each category has its own sources of error and more error in the process is caused when the assessor confuses and conflates the categories.
4. Wilson elucidates the notion of “error”: “Error is predicated on a notion of perfection; to allocate error is to imply what is without error; to know error it is necessary to determine what is true. And what is true is determined by what we define as true, theoretically by the assumptions of our epistemology, practically by the events and non-events, the discourses and silences, the world of surfaces and their interactions and interpretations; in short, the practices that permeate the field. . . Error is the uncertainty dimension of the statement; error is the band within which chaos reigns, in which anything can happen. Error comprises all of those eventful circumstances which make the assessment statement less than perfectly precise, the measure less than perfectly accurate, the rank order less than perfectly stable, the standard and its measurement less than absolute, and the communication of its truth less than impeccable.”
In other words all the logical errors involved in the process render any conclusions invalid.
5. The test makers/psychometricians, through all sorts of mathematical machinations attempt to “prove” that these tests (based on standards) are valid-errorless or supposedly at least with minimal error [they aren’t]. Wilson turns the concept of validity on its head and focuses on just how invalid the machinations and the test and results are. He is an advocate for the test taker not the test maker. In doing so he identifies thirteen sources of “error”, any one of which renders the test
making/giving/disseminating of results invalid. And a basic logical premise is that once something is shown to be invalid it is just that, invalid, and no amount of “fudging” by the psychometricians/test makers can alleviate that invalidity.
6. Having shown the invalidity, and therefore the unreliability, of the whole process Wilson concludes, rightly so, that any result/information gleaned from the process is “vain and illusory”. In other words start with an invalidity, end with an invalidity (except by sheer chance every once in a while, like a blind and anosmic squirrel who finds the occasional acorn, a result may be “true”) or to put in more mundane terms crap in-crap out.
7. And so what does this all mean? I’ll let Wilson have the second to last word: “So what does a test measure in our world? It measures what the person with the power to pay for the test says it measures. And the person who sets the test will name the test what the person who pays for the test wants the test to be named.”
In other words it attempts to measure “’something’ and we can specify some of the ‘errors’ in that ‘something’ but still don’t know [precisely] what the ‘something’ is.” The whole process harms many students as the social rewards for some are not available to others who “don’t make the grade (sic)” Should American public education have the function of sorting and separating students so that some may receive greater benefits than others, especially considering that the sorting and separating devices, educational standards and standardized testing, are so flawed not only in concept but in execution?
My answer is NO!!!!!
One final note with Wilson channeling Foucault and his concept of subjectivization:
“So the mark [grade/test score] becomes part of the story about yourself and with sufficient repetitions becomes true: true because those who know, those in authority, say it is true; true because the society in which you live legitimates this authority; true because your cultural habitus makes it difficult for you to perceive, conceive and integrate those aspects of your experience that contradict the story; true because in acting out your story, which now includes the mark and its meaning, the social truth that created it is confirmed; true because if your mark is high you are consistently rewarded, so that your voice becomes a voice of authority in the power-knowledge discourses that reproduce the structure that helped to produce you; true because if your mark is low your voice becomes muted and confirms your lower position in the social hierarchy; true finally because that success or failure confirms that mark that implicitly predicted the now self-evident consequences. And so the circle is complete.”
In other words students “internalize” what those “marks” (grades/test scores) mean, and since the vast majority of the students have not developed the mental skills to counteract what the “authorities” say, they accept as “natural and normal” that “story/description” of them. Although paradoxical in a sense, the “I’m an “A” student” is almost as harmful as “I’m an ‘F’ student” in hindering students becoming independent, critical and free thinkers. And having independent, critical and free thinkers is a threat to the current socio-economic structure of society.
LikeLike
Actually Wilson doesn’t say “vain and illusory” in his dissertation but in his review of the testing bible A Little Less than Valid: An Essay Review http://www.edrev.info/essays/v10n5.pdf
From A Little Less than Valid:
To the extent that these categorisations are accurate or valid at an individual level, these decisions may be both ethically acceptable to the decision makers, and rationally and emotionally acceptable to the test takers and their advocates. They accept the judgments of their society regarding their mental or emotional capabilities. But to the extent that such categorisations are invalid, they must be deemed unacceptable to all concerned.
Further, to the extent that this invalidity is hidden or denied, they are all involved in a culture of symbolic violence. This is violence related to the meaning of the categorisation event where, firstly, the real source of violation, the state or educational institution that controls the meanings of the categorisations, are disguised, and the authority appears to come from another source, in this case from professional opinion backed by scientific research. If you do not believe this, then consider that no matter how high the status of an educator, his voice is unheard unless he belongs to the relevant institution.
And finally a symbolically violent event is one in which what is manifestly unjust is asserted to be fair and just. In the case of testing, where massive errors and thus miscategorisations are suppressed, scores and categorisations are given with no hint of their large invalidity components. It is significant that in the chapter on Rights and responsibilities of test users, considerable attention is given to the responsibility of the test taker not to cheat. Fair enough. But where is the balancing responsibility of the test user not to cheat, not to pretend that a test event has accuracy vastly exceeding technical or social reality? Indeed where is the indication to the test taker of any inaccuracy at all, except possibly arithmetic additions?
LikeLike
I’m dealing with a similar situation in New York. Twice a year (fall and spring) I have to administer the SANDI test to my special education students with severe Autism. The test has three sections- math, reading, writing- with over 100 items each (you stop when the student gets 5 in a row in wrong). Each item generally consists of multiple questions (e.g 10 math problems or reading comprehension questions) or writing tasks that are often far above my students’ current writing ability. My students become angry and frustrated by these tasks, and I feel like a horrible person for forcing them to complete the tasks. In addition, the test must be administered one-on-one and, as such, often takes at least a week and half to complete. I end up having to test the students all day during that entire time. It’s a massive waste of valuable instructional time. In the end, the “data” I gain from this assessment does not tell me anything that I don’t already know.
In addition to the SANDI, I must administer the FAST twice per year. The FAST is essentially a shortened version of SANDI that takes slightly less time. Again, the results of the assessment don’t tell me anything I don’t know, but the assessment does cause me to spend 3-4 days of class time administering it.
In addition to SANDI and FAST, we also must administer NYSAA (New York State Alternate Assessment). For science and social studies, I need to select one of the required learning standards and collect baseline and final data in order to show growth. Even if I am not covering those standards at the moment, I must put my unit on hold to address these standards that are often meaningless to my students. For literacy and math, the students take a computer based test. This test takes several hours for students to complete. The test supposedly adapts to the students’ levels based on their answers to questions, but, for one student, the test went from solving simple addition problems in one question to asking the student to identify parallel and perpendicular lines in another. We do not address parallel and perpendicular lines. Most of my students struggle with basic addition and subtraction, so we focus heavily on those concepts. Further, some of the reading comprehension questions were so poorly written that I (a person with a masters degree in education) was unsure what the correct was. Also, the accommodations for visually impaired students were absurd. When you set the test to “large print” the print became so large that the entire question couldn’t fit on the screen. You had to continually scroll side to side to read the questions and answer choices. My students were frustrated by the entire experience. To top it all off, while teachers administered this test one-on-one, the rest of their students were shown movies in the auditorium. If that’s not a massive waste of potential instructional time, then I certainly don’t know what is. This testing had got to stop.
LikeLike