Daniel Katz of Seton Hall University pulled apart a recent editorial in the New York Times which hailed the preservation of annual testing in federal law as a great and necessary step towards improving education for all children, especially the neediest who would otherwise be overlooked. This is the same line we have heard about the virtues of No Child Left Behind since 2002, when it was signed into law. Without annual testing, would we know how any student was doing? Would we know if teachers and schools were accountable? Etc.
Without annual testing, the editorial asserts, the country would have “no way of knowing whether students are learning anything or not.” Really? Is there no one at the New York Times who has heard of the National Assessment of Educational Progress? It seems that every time a story appears in the Times, there is the same lament about how we have “no way” to know, etc., but we do! NAEP tests scientific samples of students in every state and in many cities. Why must we squander billions on testing every child every year, when the same money could be spent to reduce class sizes, to employ teachers of the arts, to hire guidance counselors, and to restore libraries?
Katz writes:
Such statements might have been viable in 2001 when the NCLB legislation was passed with bipartisan support, but after nearly a decade and half, there is no evidence to be found that test based accountability is telling us anything we did not already know from other means, nor is there evidence that the children whose plights provided NCLB’s rationale are prospering. To be honest, at this point in our policy cycle, it takes a love of annual standardized testing similar to Smeagol’s love of the One Ring to be blinded to how thoroughly it has failed to improve our schools.
He then cites the latest NAEP data to show that after 13 years of annual testing, gains have ground to a halt, and achievement gaps persist.
If we mark the NLCB era from the 2002 test administration, then we have to conclude that, in the 8th grade reading NAEP, the gap in scores between white and black students has closed a grand total of one point. The 4th grade gap has closed a more generous four points in the same time. In mathematics, the NCLB era has seen a score gap in both 4th and 8th grade close all of three points.
Paul Barton reported years ago in a study of the Black-White Achievement Gap for the Educational Testing Service that the most significant narrowing of the gap occurred from 1973-1988 (Katz includes the graphs from NAEP). Barton attributed the narrowing of the gap to racial integration, reduced class sizes, more early childhood education, and increased economic opportunities for African American families.
Yet in the era of high-stakes testing, the gap has widened and remained persistently large.
Katz writes:
There is a limited role that standardized test data can play in a comprehensive system of school monitoring, development, and accountability, but it must play a small role at best in coordination with a system that is premised on support and development. However, no school accountability system, regardless of premise, is capable of turning around a 40 year long, society spanning, trend towards inequality and segregation. That requires far more than clinging to annual, mass, standardized testing as our most vital means of giving every child access to an equitable education, and if The Times and other testing advocates really cannot see past that, then they are not merely shortsighted; they are clinging to damaging and delusional policies. A bit like our, poor, deluded Smeagol and his final cry of “Precious!”
“There is a limited role that standardized test data can play in a comprehensive system of school monitoring, development, and accountability,. . . ”
No there isn’t!
How in the hell can there be a role for an educational malpractice, standardized testing, in which any results gleaned are COMPLETELY INVALID? COMPLETELY INVALID and you still want to give standardized testing ANY ROLE??? Yep, and I’ve got that ocean front white sand beach over at Lake of the Ozarks in central Missouri to sell you cheaply.
It is the height of idiocy, of insanity, of stupidity to make any decisions on the teaching and learning process and its outcomes on the COMPLETE INVALIDITY of standardized test results. What Bill the Cat says:
And for those new here that can’t conceive of the COMPLETE INVALIDITY of standardized test scores I urge you to read Noel Wilson’s never rebutted nor refuted total takedown of educational standards and standardized testing: “Educational Standards and the Problem of Error” found at: http://epaa.asu.edu/ojs/article/view/577/700
Brief outline of Wilson’s “Educational Standards and the Problem of Error” and some comments of mine.
1. A description of a quality can only be partially quantified. Quantity is almost always a very small aspect of quality. It is illogical to judge/assess a whole category only by a part of the whole. The assessment is, by definition, lacking in the sense that “assessments are always of multidimensional qualities. To quantify them as unidimensional quantities (numbers or grades) is to perpetuate a fundamental logical error” (per Wilson). The teaching and learning process falls in the logical realm of aesthetics/qualities of human interactions. In attempting to quantify educational standards and standardized testing the descriptive information about said interactions is inadequate, insufficient and inferior to the point of invalidity and unacceptability.
2. A major epistemological mistake is that we attach, with great importance, the “score” of the student, not only onto the student but also, by extension, the teacher, school and district. Any description of a testing event is only a description of an interaction, that of the student and the testing device at a given time and place. The only correct logical thing that we can attempt to do is to describe that interaction (how accurately or not is a whole other story). That description cannot, by logical thought, be “assigned/attached” to the student as it cannot be a description of the student but the interaction. And this error is probably one of the most egregious “errors” that occur with standardized testing (and even the “grading” of students by a teacher).
3. Wilson identifies four “frames of reference” each with distinct assumptions (epistemological basis) about the assessment process from which the “assessor” views the interactions of the teaching and learning process: the Judge (think college professor who “knows” the students capabilities and grades them accordingly), the General Frame-think standardized testing that claims to have a “scientific” basis, the Specific Frame-think of learning by objective like computer based learning, getting a correct answer before moving on to the next screen, and the Responsive Frame-think of an apprenticeship in a trade or a medical residency program where the learner interacts with the “teacher” with constant feedback. Each category has its own sources of error and more error in the process is caused when the assessor confuses and conflates the categories.
4. Wilson elucidates the notion of “error”: “Error is predicated on a notion of perfection; to allocate error is to imply what is without error; to know error it is necessary to determine what is true. And what is true is determined by what we define as true, theoretically by the assumptions of our epistemology, practically by the events and non-events, the discourses and silences, the world of surfaces and their interactions and interpretations; in short, the practices that permeate the field. . . Error is the uncertainty dimension of the statement; error is the band within which chaos reigns, in which anything can happen. Error comprises all of those eventful circumstances which make the assessment statement less than perfectly precise, the measure less than perfectly accurate, the rank order less than perfectly stable, the standard and its measurement less than absolute, and the communication of its truth less than impeccable.”
In other words all the logical errors involved in the process render any conclusions invalid.
5. The test makers/psychometricians, through all sorts of mathematical machinations attempt to “prove” that these tests (based on standards) are valid-errorless or supposedly at least with minimal error [they aren’t]. Wilson turns the concept of validity on its head and focuses on just how invalid the machinations and the test and results are. He is an advocate for the test taker not the test maker. In doing so he identifies thirteen sources of “error”, any one of which renders the test making/giving/disseminating of results invalid. And a basic logical premise is that once something is shown to be invalid it is just that, invalid, and no amount of “fudging” by the psychometricians/test makers can alleviate that invalidity.
6. Having shown the invalidity, and therefore the unreliability, of the whole process Wilson concludes, rightly so, that any result/information gleaned from the process is “vain and illusory”. In other words start with an invalidity, end with an invalidity (except by sheer chance every once in a while, like a blind and anosmic squirrel who finds the occasional acorn, a result may be “true”) or to put in more mundane terms crap in-crap out.
7. And so what does this all mean? I’ll let Wilson have the second to last word: “So what does a test measure in our world? It measures what the person with the power to pay for the test says it measures. And the person who sets the test will name the test what the person who pays for the test wants the test to be named.”
In other words it attempts to measure “’something’ and we can specify some of the ‘errors’ in that ‘something’ but still don’t know [precisely] what the ‘something’ is.” The whole process harms many students as the social rewards for some are not available to others who “don’t make the grade (sic)” Should American public education have the function of sorting and separating students so that some may receive greater benefits than others, especially considering that the sorting and separating devices, educational standards and standardized testing, are so flawed not only in concept but in execution?
My answer is NO!!!!!
One final note with Wilson channeling Foucault and his concept of subjectivization:
“So the mark [grade/test score] becomes part of the story about yourself and with sufficient repetitions becomes true: true because those who know, those in authority, say it is true; true because the society in which you live legitimates this authority; true because your cultural habitus makes it difficult for you to perceive, conceive and integrate those aspects of your experience that contradict the story; true because in acting out your story, which now includes the mark and its meaning, the social truth that created it is confirmed; true because if your mark is high you are consistently rewarded, so that your voice becomes a voice of authority in the power-knowledge discourses that reproduce the structure that helped to produce you; true because if your mark is low your voice becomes muted and confirms your lower position in the social hierarchy; true finally because that success or failure confirms that mark that implicitly predicted the now self-evident consequences. And so the circle is complete.”
In other words students “internalize” what those “marks” (grades/test scores) mean, and since the vast majority of the students have not developed the mental skills to counteract what the “authorities” say, they accept as “natural and normal” that “story/description” of them. Although paradoxical in a sense, the “I’m an “A” student” is almost as harmful as “I’m an ‘F’ student” in hindering students becoming independent, critical and free thinkers. And having independent, critical and free thinkers is a threat to the current socio-economic structure of society.
I will be glad to help anyone read through and understand Wilson’s work. If you have any questions, comments or would like some help please contact me at: dswacker@centurytel.net
Who is Wilson?
Paul Krugman often writes about what he calls zombie ideas in economic policy– ideas that persist in spite of evidence. This also applies to education. Why?
(1) Because the values that support beliefs, and in turn, ideologies are extraordinarily resilient and often not subject to change based on evidence.
(2) The ideas and subsequent policies support self-interest, either direct profit or maintaining privileged positions.
(3) People in power often view admission of error as a sign of weakness, rather than wisdom.
Test-based accountability, charter schools, etc. don’t make sense or stand up to evidentiary analysis through equity, democracy and social responsibility lenses. However, claims aside, these are not the values of their promoters. The more altruistic among them are looking for a fairer shot within the framework of an inherently inequitable society.
As the struggle shifts to the states, we need to lead with our values and follow with specific solutions. We need to reframe the education debate.
Read more here: https://www.washingtonpost.com/news/answer-sheet/wp/2014/08/20/the-strategic-campaign-public-education-supporters-need-in-nine-steps/
http://www.arthurcamins.com
What is even more interesting is that the fact that the same corporate media (with an emphasis on corporate) will point out the rankings of the US on the OECD’s international PISA test comparing 60+ countries—-another test that uses the same randomized methods that the NAEP uses to measure the growth of what students are learning–and remembering—to criticize the public schools. None of these tests looks at the quality of teaching—just what children remember from what they learned.
The overall average rankings of the PISA are acceptable to criticize the public schools in the United States but the NAEP test that shows steady growth and improvements for decides is ignored as if it doesn’t exist.
Then there is the fact that the 2012 PISA tests tested too many children in the US who come from poverty causing the overall ranking to be lower than it would have been, if the proper ratios were tested.
Children who live in poverty do poorly on the PISA in EVERY country that is tested but amazingly—-a Stanford study reveled—do better in the U.S. and are improving while poor children in even the highest ranked PISA countries are doing worse. But the corporate media ignores that too.
Did you get that? The public schools in the United States are doing better teaching children who live in poverty than all the other countries PISA tests but corporate Charters suck at teaching the same students.
That NYTimes editorial, in summary:
Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test.
Ahhhh…
As Stephen Colbert reiterated on his show recently, it was Gollum and not Smeagol who was enraptured with his Precioussssss. Otherwise, a great post.
After reading other comments, I have to update my own. Using standardized test data to measure student progress means that the tests must already have been standardized prior to administration for testing purposes. Right there, this requirement negates the use of PARCC tests, which were rolled out without ever having been standardized on a normative population. They have not undergone statistical testing for content validity, as many anecdotes have shown. So aside from being conceptually invalid, they are also statistically invalid and non-standardized. Why are we using them?
“So aside from being conceptually invalid, they are also statistically invalid and non-standardized.”
Knowing the first part of your statement to be true the second part doesn’t matter (although it’s a good talking point.)
“Why are we using them?”
$$$$$$$$$$$$$$ for a few, G$%D*&^$F%$&%@# misery for the many, especially the lower class.
I’d like to read in the NYT, why urban children benefit from 266% more testing than their suburban counterparts. A minority student explained, in Mother Jones, why they don’t.
“Without annual testing, would we know how any student was doing? Would we know if teachers and schools were accountable?”
There is an element and a role NCLB, the forthcoming ESEA, and ultimately, the Common Core play in education. The profession is paid lip service at best in certain circles. Teachers deserve better pay, smaller class sizes, and meaningful professional development. Standardized testing results establish a front page score that serves as a barometer of substance. The general public, many parents, and truthfully, most legislators will never set foot in the classroom. College graduates are not lining up to teach- it’s a career that requires skill, passion, and sacrifice. I’m against tying test scores to teacher evaluations in an all or nothing manner, but a segment tied to pay raises is not all bad. Healthy competition builds success. As a teacher, I compete every day- for my student’s attention, my lessons need to engage; to meet standards, the Common Core; and to assess my students with meaningful feedback. A uniform barometer is important to garner respect, improve instructional skill, and graduate College and Career Ready citizens.
“As a teacher, I compete every day- for my student’s attention, my lessons need to engage; to meet standards, the Common Core; and to assess my students with meaningful feedback.”
As teachers, we do NOT need Common Core and its standardized tests to assess students. I taught for thirty years 1975-2005 and most of those years were without the gulag of standardized tests that have plagued the schools like a witch’s curse since at least 1999.
Every assignment students turned in and I corrected offered me an opportunity to assess how my students were learning or not. I knew who wasn’t working because they didn’t turn anything in. I knew who didn’t understand the work because the results of the work they turned in revealed to me that fact. This list is endless. Most if not all teachers I worked with also did the same thing.
If a lesson was a dismal failure due to the results of the work, I’d reteach the lesson using different methods.
Without standards I knew my students needed to learn how to spell correctly, learn grammar and mechanics and how to understand all the elements of literature and the difference between the genres in addition to learning out to write properly. Without any Common Core Crap standards I taught my students what would improve their reading and writing literacy.
The lessons I designed without any help from any sort of Common Core Crap were always written to teach and capture the attention of as many students as possible. And when one of my lessons worked really well, I shared with my deparemtn what I had done just like they shared their success with the department.
Long before NCLB, RTTT, CCC and high stakes rank and punishment came along, we didn’t not teach in a vacumn. We knew what our students should learn and we taught what had to be taught.
And I never taught in isolation. I had my department team. We met as a department and also in grade level teams to discuss what our students needed most and share methods that worked. When I first started teaching under a full time contract in 1978-79, my department chair offered advice and material in addition to other teachers in the English department. Even as a substitute for the two years I did that before being hired into a full time position, teachers and administrators often offered advice, support and help.
The Common Core Crap is nothing but a tool designed to destroy the community based, non-profit, transparent, democratic public schools. And any one who thinks otherwise is either a fool for being fooled or a shill for the for the for-profit, autocratic, opaque, often fraudulent and abusive of children and teachers corporate education industry.