In a recent post, I criticized Alabama for setting goals based on race, ethnicity, and disability status. I said it was unAmerican. Our goal as a society is equality of educational opportunity. There is something repulsive, to me at least, in saying that schools will set targets based on the color of children’s skin, their parents’ income, or other factors. We know that not all kids will end up at the same point by the end of each year, but we should not predetermine what we expect. I think the goal should be to treat each child as a unique human being and be sure they have the opportunity and resources they need to get a sound education.
But I must apologize to Alabama. Other states have similar race-based, ethnicity-based, disability-based goals.
Apparently they do this to satisfy the requirements of the federal government, either NCLB or the Obama waivers.
Why is the government setting targets for test scores? A standardized test should be used–if at all–diagnostically, to identify what kind of extra help students need. Instead, states are trapped in stale NCLB thinking. It hasn’t worked for 12 years. Why expect that tinkering will fix what is inherently wrong?
Stop measuring with a broken stick. Standardized tests are one indicator. Turning them into the be-all and end-all of schooling is wrong. It corrupts education. It causes otherwise thoughtful people to expect more of the tests than they can deliver. We need better goals than test scores. By relying on them so much, we sacrifice qualities that matter far more and debase schooling.
This is accountability run amok. This is the kind of policy that should be openly discussed and debated. We cannot allow it to be institutionalized and made permanent. It is an embarrassment to our democracy.
Diane Ravitch's blog 
Thank you Diane. So well put. It is this odd obsession with “standardized test learning growth” that leads to these perverse systems.
LikeLike
carolcorbettburris: so well put!
“It is this odd obsession with ‘standardized test learning growth’ that leads to these perverse systems.”
That is why I have ceased thinking about high-stakes standardized testing as leading to “teaching to the test” but rather inclining ever more strongly to the more pernicious “teaching the test.” And this practice literally subverts the sampling principle of domains that undergirds standardized testing as the psychometricians themselves say it should be done.
“The soft bigotry of low expectations”? What you describe is the “hard bigotry of mandated failure.”
Thank you for your posting.
🙂
LikeLike
“. . . standardized test learning growth. . . ”
Now there’s an oxymoronic concept if I’ve ever seen one!!
LikeLike
In training to be a gifted Ed teacher at KU,I remember having to memorize the bell curve for each race, because the average IQ scores( as recorded per established measurement tests in the early 2000s) differed. Should it be that this type data is absorbed but not mentioned, or should this type data not exist because the measurement tests are biased by the very nature of how our diverse culture came about?
LikeLike
I am against rampant testing, but aren’t these STATE goals and not district or school goals?
The big fear is that teachers will write off black students because “less of them are expected to pass.” However, that is not true inasmuch these are state goals and will not impact VAM scores.
LikeLike
Again, since this testing stuff seems inevitable, it would have been better to do it based on socioeconomics. However, that would then dispell the reformers’ belief that poverty doesn’t matter.
I say give us all the same education that the politicians’ kids get. Especially the kids of “reformers.” They do a good job of exempting themselves from all of the policies they impose on the masses.
LikeLike
Actually, in Alabama the increases expected for those groups with lower lower initial scores are much higher than those with higher initial scores. The state is trying to keep from being punished for the current state of affairs before they have a chance to do anything (not that they are likely to succeed but that is another story)
The repugnant thing is that Alabama decided to separate african americans from mixed-race children. They are so focused on race-absing this that they are inventing classifications that probably don’t exist.
LikeLike
Yes they are state goals, but if schools do not meet them they risk losing funding, so individual districts and schools are forced to comply with bad policies from the top.
LikeLike
Teachers are “against rampant testing” until it comes down to them serving as counselors, or getting a day off from teaching by administering tests, or getting extra pay from testing companies for creating tests.
For the past 40 years they never speak up against testing, especially invalid, inaccurate and poor quality testing for a multitude of selfish reasons. Thus they have squandered their credibility.
And we don’t even need to mention that most of them have done so badly on the ACTs and SATs they took, that they performed so shamefully on the GREs for their entrance in MEd programs such that their second-rate schools won’t even cooperate with studies asking for those University’s Master’s Programs average GRE scores.
“A standardized test should be used–if at all–diagnostically, to identify what kind of extra help students need.” It is forty years to late to try to claim the high ground in this. Besides, you don’t want to jeopardize your chances of advancement into school administration by letting your superiors know you believe such heresy.
Methinks thou doth protesteth too much.
LikeLike
I’ve never heard a teacher get excited to have a day “off” to grade a bunch of standardized tests. Getting a sub is a pain, and we would rather be with our kids than looking at a bunch of letters.
I think YOU protest too much, Michael.
LikeLike
I don’t know whether your summary of the general mind set of the cadre of public school teachers is true or not, but IF it is true, it is damning. One of my public school teacher friends once said to me, “I hate using scantrons, but it’s so much easier.” I said nothing. I hadn’t walked in her moccasins.
LikeLike
Michael,
What were your SAT, ACT and/or GRE Scores?
Bet mine were higher-ha ha!d
LikeLike
HU,
“I don’t know whether your summary of the general mind set of the cadre of public school teachers is true or not,. . . ”
It’s not!
But I will agree that way too many teachers and administrators have “gone along to get along” by instituting the various “reforms” that they know are harmful to students and have not spoken out against them (probably because they have seen what happens to someone like me when we don’t accept the changes that we know are harmful to students).
LikeLike
Michael,
I don’t get it. Reformers complain low SAT/ACT scores from College of Education students but at the same time talk about teacher certification being “an unnecessary loophole” for people to enter the profession. Then you have some places where they don’t even care if you have a degree if you want to teach.
LikeLike
Thanks Diane for criticizing race-based goal-setting. Having experienced the perils of this trend for too long in my building I couldn’t agree more. As for institutionalizing testing, I agree. That is the battle we must fight. The problem with standardized testing accountability is that it triggers a near universal response of “If the kids are achieving, shouldn’t the test show it?” We know this kind of simple thinking is erroneous, but it works beautifully to the advantage of corporate reformers. Challenging this knee jerk reaction by the non-educators is what we must confront to get at the root of the problem.
LikeLike
“. . . to get at the root of the problem.”
The root of the problem is that educational standards and standardized testing are logically invalid and no amount of psychometric fudging can overcome all the logical errors involved in the process of making standards, the giving and taking of standardized tests, and the disseminating of the results which render the process invalid as shown by Noel Wilson in “Educational Standards and the Problem of Error” found at: http://epaa.asu.edu/ojs/article/view/577/700 . See below for a summary and my comments:
A brief outline of Wilson’s “Educational Standards and the Problem of Error” and some comments of mine. (updated 6/24/13 per Wilson email)
1. A quality cannot be quantified. Quantity is a sub-category of quality. It is illogical to judge/assess a whole category by only a part (sub-category) of the whole. The assessment is, by definition, lacking in the sense that “assessments are always of multidimensional qualities. To quantify them as one dimensional quantities (numbers or grades) is to perpetuate a fundamental logical error” (per Wilson). The teaching and learning process falls in the logical realm of aesthetics/qualities of human interactions. In attempting to quantify educational standards and standardized testing we are lacking much information about said interactions.
2. A major epistemological mistake is that we attach, with great importance, the “score” of the student, not only onto the student but also, by extension, the teacher, school and district. Any description of a testing event is only a description of an interaction, that of the student and the testing device at a given time and place. The only correct logical thing that we can attempt to do is to describe that interaction (how accurately or not is a whole other story). That description cannot, by logical thought, be “assigned/attached” to the student as it cannot be a description of the student but the interaction. And this error is probably one of the most egregious “errors” that occur with standardized testing (and even the “grading” of students by a teacher).
3. Wilson identifies four “frames of reference” each with distinct assumptions (epistemological basis) about the assessment process from which the “assessor” views the interactions of the teaching and learning process: the Judge (think college professor who “knows” the students capabilities and grades them accordingly), the General Frame-think standardized testing that claims to have a “scientific” basis, the Specific Frame-think of learning by objective like computer based learning, getting a correct answer before moving on to the next screen, and the Responsive Frame-think of an apprenticeship in a trade or a medical residency program where the learner interacts with the “teacher” with constant feedback. Each category has its own sources of error and more error in the process is caused when the assessor confuses and conflates the categories.
4. Wilson elucidates the notion of “error”: “Error is predicated on a notion of perfection; to allocate error is to imply what is without error; to know error it is necessary to determine what is true. And what is true is determined by what we define as true, theoretically by the assumptions of our epistemology, practically by the events and non-events, the discourses and silences, the world of surfaces and their interactions and interpretations; in short, the practices that permeate the field. . . Error is the uncertainty dimension of the statement; error is the band within which chaos reigns, in which anything can happen. Error comprises all of those eventful circumstances which make the assessment statement less than perfectly precise, the measure less than perfectly accurate, the rank order less than perfectly stable, the standard and its measurement less than absolute, and the communication of its truth less than impeccable.”
In other word all the errors involved in the process render any conclusions invalid.
5. The test makers/psychometricians, through all sorts of mathematical machinations attempt to “prove” that these tests (based on standards) are valid-errorless or supposedly at least with minimal error [they aren’t]. Wilson turns the concept of validity on its head and focuses on just how invalid the machinations and the test and results are. He is an advocate for the test taker not the test maker. In doing so he identifies thirteen sources of “error”, any one of which renders the test making/giving/disseminating of results invalid. As a basic logical premise is that once something is shown to be invalid it is just that, invalid, and no amount of “fudging” by the psychometricians/test makers can alleviate that invalidity.
6. Having shown the invalidity, and therefore the unreliability, of the whole process Wilson concludes, rightly so, that any result/information gleaned from the process is “vain and illusory”. In other words start with an invalidity, end with an invalidity (except by sheer chance every once in a while, like a blind and anosmic squirrel who finds the occasional acorn, a result may be “true”) or to put in more mundane terms shit-in shit out.
7. And so what does this all mean? I’ll let Wilson have the second to last word: “So what does a test measure in our world? It measures what the person with the power to pay for the test says it measures. And the person who sets the test will name the test what the person who pays for the test wants the test to be named.”
In other words it measures “something” which supposedly is specified by the test maker but the whole process is so error ridden that any conclusions drawn are invalid. The test supposedly measures “’something’ and we can specify some of the ‘errors’ in that ‘something’ but still don’t know [precisely] what the ‘something’ is.” The whole process harms many students as the social rewards for some are not available to others who “don’t make the grade (sic)” Should American public education have the function of sorting and separating students so that some may receive greater benefits than others, especially considering that the sorting and separating devices, educational standards and standardized testing, are so flawed not only in concept but in execution?
My answer is NO!!!!!
One final note with Wilson channeling Foucault and his concept of subjectivization:
“So the mark [grade/test score] becomes part of the story about yourself and with sufficient repetitions becomes true: true because those who know, those in authority, say it is true; true because the society in which you live legitimates this authority; true because your cultural habitus makes it difficult for you to perceive, conceive and integrate those aspects of your experience that contradict the story; true because in acting out your story, which now includes the mark and its meaning, the social truth that created it is confirmed; true because if your mark is high you are consistently rewarded, so that your voice becomes a voice of authority in the power-knowledge discourses that reproduce the structure that helped to produce you; true because if your mark is low your voice becomes muted and confirms your lower position in the social hierarchy; true finally because that success or failure confirms that mark that implicitly predicted the now self evident consequences. And so the circle is complete.”
In other words students “internalize” what those “marks” (grades/test scores) mean, and since the vast majority of the students have not developed the mental skills to counteract what the “authorities” say, they accept as “natural and normal” that “story/description” of them. Although paradoxical in a sense, the “I’m an “A” student” is almost as harmful as “I’m an ‘F’ student” in hindering students becoming independent, critical and free thinkers. And having independent, critical and free thinkers is a threat to the current socio-economic structure of society.
LikeLike
Alright, this is by far the lengthiest, most interesting and valuable information laden comment I’ve ever received! Thank you so much! Can you send me a source link to this work by Wilson? I need to read it in full. You have expertly shown the illogical reasoning behind testing. When it comes to race-based thinking, this is all the more true. In this regard, the most preposterous thing I have experienced in my career is the categorizing of children by racial group regardless of community and cultural background. Case in point, the African-American kids are combined with the Ethiopian and Nigerian immigrant kids under the heading “Black”. Do you know how much Nigerians and Ethiopians have in common? Aside from coming from the same continent, that’s about it. Moreover, neither group has anything to do with African-Americans. Fueling such ludicrous, and frankly racist groupings is the wholesale ignorance on the part of teachers & administrators of these children’s backgrounds. I would like to continue corresponding with you. Please follow my blog if you haven’t already, and follow me on FB too if you’re there. Thanks again!
LikeLike
Iain,
Thanks for the kind words about my comment. I post this often (almost daily) as Wilson’s work is, in my mind, truly THE most important educational policy analysis of the last 50 years. Until and unless all educators read, understand and act on the findings of the study, i.e., the complete invalidity and anti-egalitarian aspect of sorting and separating students via educational standards, standardized testing and the “grading of students, many a student will be harmed. And that is a travesty, an abomination, and is truly insane. (Yeah, I’m calling out all you educators in the crowd as being truly insane and causing harm to your students.)
The link to the work is in my response above. For good reading see Arizona State University’s “Educational Policy Analysis Archives” online. Another excellent piece of Wilson’s work is “A Little Less than Valid: An Essay Review” found at: http://www.edrev.info/essays/v10n5.pdf .
If in reading Wilson’s work you have questions or concerns you can contact me at: dswacker@centurytel.net . I have been in contact with Neil and he wishes that more people would read and understand his work!
Again, thanks for the kind words,
Duane Swacker
(aka ‘old fart’ public high school Spanish teacher)
LikeLike
I agree with the post that generated this discussion and I think many an attempt to be kind and fair has led to profound insensitivity to the realities of individual human being and terrible miscalculations regarding their abilities and, for that matter, their worth. While I understand the value of grouping people together on the basis of a set of variables for the sake of convenience for research projects and decision making in various institutions, when dealing with human beings, what is ignored is exactly what makes a human being human, his or her unique intellect which, by its very nature, produces unique prospectives on the meaning of life on this planet, in this universe. It is the uniqueness of the individual human being that is the foundation for the democratic process, one in which the unique opinions of individuals provide the material for the societal dialect that leads to policies that are sensible and for the good of the whole and, ultimately, all of its individual members. Social sciences, almost by nature, are reductionist, and to an extent too great to be tolerable in many situations, by their nature are dehumanizing. Doing away troublesome variables that are troublesome because they are the product of individuality, makes data processing possible, manageable. And such data leads to conclusions that lead to policies intended to manage people, almost never to liberate the unique individual intellect. Instead of instruction, too many involved in education deliver programs, programs based on good science BUT rarely upon what is truly good for individual growth and development of an individual’s potential. I think that a scientific approach is essential, but I think we need a very different kind of science than the one we apply now. We need science infused with humanity, not a sloppy science lacking in rigor and reason, but one that does not dismiss critically important variables in favor of easy generalizations that are never easy on those they fail to take into consideration.
LikeLike
Thanks you Diane. I live in Alabama and was horrified when I heard about the new education policies. Unfortunately, I will also have to live it when returning to work in the Fall. But like you, I was even more upset when I saw this is a national trend. I hope that I am not in the minority when I say that I am going to teach in the best interest of my students regardless of these racist policies.
LikeLike
As Diane suggests, this is just a side-effect of using standardized testing for accountability purposes rather than diagnostics. Once you attach consequences to the tests, there’s no way to win when it comes to race. If a state varies its targets by race and ethnicity, then people will criticize the policy as racist. If a state doesn’t vary its targets by race and ethnicity, then people will say the policy is racist when disproportionate numbers of minorities fail to meet the targets. If a state tries to devise more complex targets based on multiple factors, then people will say the policy is based on junk science. We’ve seen all these scenarios play out like this already.
LikeLike
FLERP!,
“. . . this is just a side-effect of using standardized testing for accountability purposes rather than diagnostics.”
They’re not any good at diagnostics either considering all the epistemological and ontological problems that standardized tests entail (see Wilson’s work referenced above). Epistemologically and ontologically speaking: Psychometrics = Phrenology = Eugenics = Blood Letting.
LikeLike
Wow, this really is an embarrassment. Why stop at race? Gender goals? Socioeconomic status goals? Is there any misuse to which standardized tests cannot be put?
LikeLike
Standardized tests as a dressing don’t do much to enhance your salad. So there’s one misuse!
LikeLike
Wonderful remarks throughout the blog and much to learn from and utilize. However, all of it seems to come down to the use of common sense,reason, and collective courage.
As an advocate for the disabled I remember going to a school to represent a child with Cancer. She was wearing a scarf wrapped around her head to cover her now baldness from the chemo treatments she was receiving. Her parents were told that she could not wear the scarf as it was against the school policy. As I sat with the teacher and child study team I could not but be struck by the lack of flexibility for common sense and reason. I was told that she could not wear the scarf because no child was allowed to have anything on their heads according to the policy and if they gave her permission it would cause a problem with the other children.
You all know my argument was that she was not like the other children as they were not going through chemo and this scarf was an accomodation to ensure her health, welfare, and safety. It was written into her IEP and all were satisfied. The fear by the school participants for their own concerns of sameness and strict adherence to a zero tolerance policy for anything was ludicrous and defied common sense and humanity. They were lock step in following each other first and what was reasonable and what was right for the child being denied. Measuring children with standarized tests no matter their differences for the purpose that seems less to do with the educational welfare of the children and more for the purposes of others is wrong. There is no truth in these practices or common sense. At least not for the children, who by the way, are being treated as lab rats. At what point does common sense and the integrity of the professionals say no more and refuse in a national rebuke of giving these tests? What else can happen to schools when they are already being closed and good practices abandoned? Unions, parents, administrations need to take ownership over their own
responsiblity to Do No Harm for the sake of the children.
LikeLike
ETS (AIR and participating groups) sponsored a forum on some of these issues and produced a bibliography in 2011. Among the conclusions: (copyright ETS)
“Little is known about using student growth as a component in teacher evaluation . This is the case for all students, but it is even more so for students with disabilities .” “Because of the limited research and the challenges involved with measuring the academic growth of students with disabilities, we caution against using results for high-stakes decisions until further research and practical experience support the validity of claims made from the various measures .”
ETS assisted with very practical standard setting and testing/assessment procedures in Massachusetts in preparation for implementing MCAS etc. This was true BEFORE Anrig went there and after his time there; ETS provided valuable assistance to 35 school districts in these measures (see ETS standard setting manual in ERIC).
The bibliography provided from this Forum 2011 is essential for those of us in teacher preparation for special educators and I will be providing copies to colleagues and students in the Greater Boston area so thought that others on the list might want this information. As I have mentioned in other posts here I followed the development of MCAS through all its previus revisions; we need to be very informed and responsive to the implications of new measures of teacher effectiveness especialy for teachers of students with disabilities (whether or not participating in general education and/or using alternate assessment methods as designated in IEPs). To quote the ETS forum again (without infringing on their copyright) measuring student growth and attributing that growth accurately for the purpose of teacher evaluation is not an easy task .” I think we have learned that in Massachusetts but the barrage from political groups is relentless and it seems to be opposed to the university system as we know it from a research perspective. ” The reading wars and the math wars were minor in comparison to this ideological attack on public education (and the relentless bashing of teachers). You can find the ETS forum report from 2011 through google …. but I will append the reference here in a comment posting.
LikeLike
reference: “Using Student Growth to Evaluate Educators of Students With Disabilities: Issues, Challenges, and Next Steps” january 2012 (summary of the 2011 forum) ….. the groups sponsoring this forum CSSO, ETS, and the NCCTQ (please note this is not the Kate Walsh affiliated group but the acronyms easily get mixed up)…
This quote is from their bibliography:
“Because the use of schoolwide or group value-added models is relatively new within the context of teacher evaluation, the availability of research is limited .”
As I mentioned I had great respect for Greg Anrig in his implementation and policies for Massachusetts in special education (766) and early developments of MCAS (in the iterative versions) and we worked independently with ETS staff on standard setting before Greg Anrig went there so my opinions about the quality of ETS’ staff development in our school districts do not represent a conflict of interest . ETS manual on standard setting was a valuable step in the process for our school districts and I am concerned for the teachers in states that jump immediately into elaborate computer and commercial driven testing with out the preliminary steps that we went through over these years with staff development. I never said MCAS was perfect but I worry for the states and teachers that have this foisted on them through political machinations…. (Deus ex machina)
LikeLike
No need to apologize to Alabama. We should have never gone there in the first place. This is the first time I have seen different cut scores for groups in Alabama. There are so many things wrong with this and I defer to all the other great posters that have nailed the problems previously.
I especially agree with lafered who wrote in a comment:
“Social sciences, almost by nature, are reductionist, and to an extent too great to be tolerable in many situations, by their nature are dehumanizing. Doing away with troublesome variables that are troublesome because they are the product of individuality, makes data processing possible, manageable.
And such data leads to conclusions that lead to policies intended to manage people, almost never to liberate the unique individual intellect.
Instead of instruction, too many involved in education deliver programs, programs based on good science BUT rarely upon what is truly good for individual growth and development of an individual’s potential. I think that a scientific approach is essential, but I think we need a very different kind of science than the one we apply now. We need science infused with humanity, not a sloppy science lacking in rigor and reason, but one that does not dismiss critically important variables in favor of easy generalizations that are never easy on those they fail to take into consideration.”
LikeLike
I totally agree with you Diane, that each student needs to be considered as an individual, not as part of a group with separate goals, etc.
Yet, I can understand why they do that, although it is wrong.
Let us compare with affirmative action. The opponents of it say that race should not be a factor in admission to universities, hiring, etc. There should not be race-based quotas, with each individual judged on their own merits.
Theoretically that argument is correct. In a perfect world. Race should not be a factor at all. But due to the history of people of color being discriminated against in hiring, college admissions, etc., it has been felt that affirmative action has been necessary to right that wrong. Probably most of us reading here agree that affirmative action is necessary, although it shouldn’t be.
There has also been a historical problem of schools, districts, administrators, teachers, and in some cases parents, having too low expectations of children of color. Even moer so in a state like Alabama. It is true also, that kids will live down to low expectations.
So I would guess that this policy is thought of kind of like affirmative action, in trying to raise expectations for different groups that traditionally have scored lower, by means of quotas.
I totally agree with you that this policy is wrong.(unlike affirmative action, although that would not be needed in a perfect world) and misguided. Just trying to guess what might be the rationale behind such a policy.
LikeLike