Georg Lind is an educational researcher and professor of psychology in a German university who has studied the moral implications of standardized testing. His bio is at the end of this post. He sent me the following short essay on the negative consequences of standardized testing:
Leviathan: The Anti-Democratic Effect of High-Stakes Tests.
We ought to think about high stakes tests in wider contexts than we usually do, namely in the context of human functioning and in the context of human rights and democracy:
(a) All tests which are based on classical test theory (CTT) and its off-springs (e.g., item-response-theory, Rasch-scaling) are essentially statistical artifacts. Their hidden psychology is at odds with our knowledge of psychological processes underlying human behavior. These tests are built on a false postulate which says: each and every human response to a test is determined only by one disposition, namely the competence or personality under consideration, except for some degree of random measurement error which can be easily minimized by repeating measurements.
This core postulate is totally wrong: A single response is usually determined but by several dispositions at the same time, not just by one. Hence a single response is ambiguous and does not allow to make any inference on a particular disposition. If data falsify this believe they are misclassified as “unreliability.” Besides, repeated measurement is virtually not possible with human subjects. Repeated questions have to be varied, and the more varied tasks are used to reduce “unreliability,” the less valid a test becomes.
Better methodologies exist, especially for the measurement and improvement of curricula and teaching methods (see my reading suggestion below). We can single out the disposition(s) determining a person’s responses only with experimentally designed tests that let us observe pattern of responses to carefully arranged pattern of tasks. Of course, such tests require much expertise and money, probably more than the private test industry is able to provide.
(b) High-stakes testing violates human rights and undermines democracy. The frequent evaluation – year by year, month by month, day by day, and sometimes even hour by hour – of students violates their basic rights and, indirectly, also of the rights of their teachers and parents. This inhumane practice has nothing to do with well reasoned and well designed assessments required before taking over a responsible position in our society. There should be more such assessments. Why don’t we examine future parents whether they are prepared well enough to raise children? This would spare us a lot of juvenile delinquency and broken up families. Or assess future politicians’ ability to run a town, a state, or a country? You can imagine what this would spare us.
Frequent high-stakes testing is also a threat to democracy. It restricts students’ thinking and reflection. It leaves too little opportunity for the development of moral competence. It produces “subjects” not citizens of a democracy. As many decades of research into the development of moral competence shows, simply through the extreme proportion of time absorbed by the preparation for evaluations and other activities required by authorities, students are prevented from developing the ability to solve problems and conflicts through thinking and discussion instead of through violence, deceit and power. They will later, as adults, depend, as Thomas Hobbes has pointed out, on a “strong state” and on dictators to keep violence, deception and power within bounds. Morally competent citizens don’t need a “Leviathan.”
Reading suggestion: “How to Teach Morality. Promoting Deliberation and Discussion. Reducing Violence and Deceit” by Georg Lind (Logos publisher, Berlin, 2016)
—————
Contact:
Dr. Georg Lind
Schottenstr. 65
78462 Konstanz, Germany
Georg.Lind@uni-konstanz.de
http://www.uni-konstanz.de/ag-moral/
Vita:
Prof. emeritus of the University of Konstanz, Department of Psychology
Doctorates in social sciences and in philosophy; master degree in psychology.
Long-time educational researcher and test developer.
Main area of research and teaching: Moral-democratic competence development and education.
Visiting professor at the University of Illinois/Chicago, Monterrey/Mexico, and Berlin/Germany.
Guest lectures and workshop-seminars in several countries, e.g., Brazil, Chile, China, Colombia, Italy, Mexico, Poland, Switzerland.
Married and three children (two adopted in Chicago)
Born in 1947.
Just added Lind’s book to my to-read list. I was encouraged by the following post about how many higher ed schools are no longer requiring the SAT/ACT for admission.
I would like to know if there are any studies about the correlation between high-stakes testing and cheating and how this varies between different educational systems. For example, I’m thinking about China with its high school exit exam, France with its baccalaureate exams, and many countries where the educational norm is to determine a student’s passing of a class through a single exam. I would also like to know how this looks with a high stakes standardized test vs. a high stakes classroom test.
I’m thinking too about the actual PROGRAMS/personnel put together and sent into low-scoring schools for helping teachers and students learn how to “beat” college entrance exam. This isn’t education-based, it is beat-the-statistical-odds based.
Reblogged this on Mark's Text Terminal.
Please provide the reference described in “Better methodologies exist, especially for the measurement and improvement of curricula and teaching methods (see my reading suggestion below).” If no better methodologies can be referenced, then this argument would be hot air.
Read Lind’s full article
Chris,
See my post below.
Cuba has better methodologies , with out arguments .
I’m sorry for whoever said said that but I disagree completely, why? Right now there is no difference between the democracy in this country than the democracy in Cuba. First I graduated from the best navy academy in the world, because people here who don’t have gpa they’re lost in the ocean. If you don’t pass that test you lose those 5 years you’ve been training for, because the last results prove who the person is , because in the final tests you can’t cheat, in class you can. Last but not least THE WORST MOVEMENT ARE CHARTER SCHOOLS, kids who come in from Cuba in 5th grade know more than a student in 12th grade in a charter school. There are 2 types of dictatorships, the one that says and admits is a dictator ship and the one that hides themselves and doesn’t say they are dictators. I believe in the old education with no computers .
Reply
“Better methodologies exist, especially for the measurement. . .”
Ummm, No those “better” measurement methodologies don’t exist. Tell me how one can be better at “measuring” the non-observable with non-existent measuring devices that aren’t calibrated against a non-existent standard unit of learning?
Measuring the non-observable is what is being attempted with standardized tests. How insane is that?
Richard Phelps, a staunch standardized test proponent (he has written at least two books defending the standardized testing malpractices) in the introduction to “Correcting Fallacies About Educational and Psychological Testing” unwittingly lets the cat out of the bag with this statement (notice how he is trying to assert by proximity that educational standardized testing and the testing done by engineers are basically the same, in other words a “truly scientific endeavor”):
“Physical tests, such as those conducted by engineers, can be standardized, of course [why of course of course], but in this volume , we focus on the measurement of latent (i.e., nonobservable) mental, and not physical, traits.” [my addition]
Now since there is no agreement on a standard unit of learning and there is no measuring device calibrated against said non-existent standard unit, how is it possible to “measure the nonobservable” which is what all this standardized testing insanity, truly insanity if you think about it, is about???
So much harm to so many students is caused by the educational malpractices that are standards and testing or as Phelps contends in “measuring the nonobservable”.
How insane is this all???
Utterly beyond my comprehension!!!
“So much harm to so many students is caused by the educational malpractices that are standards and testing or as Phelps contends in “measuring the nonobservable”.
Yeah going crazy quoting myself today! However, the fact remains that to use the results of the COMPLETELY INVALID process of educational standards and testing for anything is the definition of inane and insane. That invalidity has been proven by Noel Wilson in his never refuted nor rebutted “Educational Standards and the Problem of Error” found at: http://epaa.asu.edu/ojs/article/view/577/700
Brief outline of Wilson’s “Educational Standards and the Problem of Error” and some comments of mine.
A description of a quality can only be partially quantified. Quantity is almost always a very small aspect of quality. It is illogical to judge/assess a whole category only by a part of the whole. The assessment is, by definition, lacking in the sense that “assessments are always of multidimensional qualities. To quantify them as unidimensional quantities (numbers or grades) is to perpetuate a fundamental logical error” (per Wilson). The teaching and learning process falls in the logical realm of aesthetics/qualities of human interactions. In attempting to quantify educational standards and standardized testing the descriptive information about said interactions is inadequate, insufficient and inferior to the point of invalidity and unacceptability.
A major epistemological mistake is that we attach, with great importance, the “score” of the student, not only onto the student but also, by extension, the teacher, school and district. Any description of a testing event is only a description of an interaction, that of the student and the testing device at a given time and place. The only correct logical thing that we can attempt to do is to describe that interaction (how accurately or not is a whole other story). That description cannot, by logical thought, be “assigned/attached” to the student as it cannot be a description of the student but the interaction. And this error is probably one of the most egregious “errors” that occur with standardized testing (and even the “grading” of students by a teacher).
Wilson identifies four “frames of reference” each with distinct assumptions (epistemological basis) about the assessment process from which the “assessor” views the interactions of the teaching and learning process: the Judge (think college professor who “knows” the students capabilities and grades them accordingly), the General Frame-think standardized testing that claims to have a “scientific” basis, the Specific Frame-think of learning by objective like computer based learning, getting a correct answer before moving on to the next screen, and the Responsive Frame-think of an apprenticeship in a trade or a medical residency program where the learner interacts with the “teacher” with constant feedback. Each category has its own sources of error and more error in the process is caused when the assessor confuses and conflates the categories.
Wilson elucidates the notion of “error”: “Error is predicated on a notion of perfection; to allocate error is to imply what is without error; to know error it is necessary to determine what is true. And what is true is determined by what we define as true, theoretically by the assumptions of our epistemology, practically by the events and non-events, the discourses and silences, the world of surfaces and their interactions and interpretations; in short, the practices that permeate the field. . . Error is the uncertainty dimension of the statement; error is the band within which chaos reigns, in which anything can happen. Error comprises all of those eventful circumstances which make the assessment statement less than perfectly precise, the measure less than perfectly accurate, the rank order less than perfectly stable, the standard and its measurement less than absolute, and the communication of its truth less than impeccable.”
In other words all the logical errors involved in the process render any conclusions invalid.
The test makers/psychometricians, through all sorts of mathematical machinations attempt to “prove” that these tests (based on standards) are valid-errorless or supposedly at least with minimal error [they aren’t]. Wilson turns the concept of validity on its head and focuses on just how invalid the machinations and the test and results are. He is an advocate for the test taker not the test maker. In doing so he identifies thirteen sources of “error”, any one of which renders the test making/giving/disseminating of results invalid. And a basic logical premise is that once something is shown to be invalid it is just that, invalid, and no amount of “fudging” by the psychometricians/test makers can alleviate that invalidity.
Having shown the invalidity, and therefore the unreliability, of the whole process Wilson concludes, rightly so, that any result/information gleaned from the process is “vain and illusory”. In other words start with an invalidity, end with an invalidity (except by sheer chance every once in a while, like a blind and anosmic squirrel who finds the occasional acorn, a result may be “true”) or to put in more mundane terms crap in-crap out.
And so what does this all mean? I’ll let Wilson have the second to last word: “So what does a test measure in our world? It measures what the person with the power to pay for the test says it measures. And the person who sets the test will name the test what the person who pays for the test wants the test to be named.”
In other words it attempts to measure “’something’ and we can specify some of the ‘errors’ in that ‘something’ but still don’t know [precisely] what the ‘something’ is.” The whole process harms many students as the social rewards for some are not available to others who “don’t make the grade (sic)” Should American public education have the function of sorting and separating students so that some may receive greater benefits than others, especially considering that the sorting and separating devices, educational standards and standardized testing, are so flawed not only in concept but in execution?
My answer is NO!!!!!
One final note with Wilson channeling Foucault and his concept of subjectivization:
“So the mark [grade/test score] becomes part of the story about yourself and with sufficient repetitions becomes true: true because those who know, those in authority, say it is true; true because the society in which you live legitimates this authority; true because your cultural habitus makes it difficult for you to perceive, conceive and integrate those aspects of your experience that contradict the story; true because in acting out your story, which now includes the mark and its meaning, the social truth that created it is confirmed; true because if your mark is high you are consistently rewarded, so that your voice becomes a voice of authority in the power-knowledge discourses that reproduce the structure that helped to produce you; true because if your mark is low your voice becomes muted and confirms your lower position in the social hierarchy; true finally because that success or failure confirms that mark that implicitly predicted the now self-evident consequences. And so the circle is complete.”
In other words students “internalize” what those “marks” (grades/test scores) mean, and since the vast majority of the students have not developed the mental skills to counteract what the “authorities” say, they accept as “natural and normal” that “story/description” of them. Although paradoxical in a sense, the “I’m an “A” student” is almost as harmful as “I’m an ‘F’ student” in hindering students becoming independent, critical and free thinkers. And having independent, critical and free thinkers is a threat to the current socio-economic structure of society.
Can Trump’s behavior be explained this way? Or take other examples for some thought experiments.
I was struck by his reference to Hobbes, as we are just finishing the part of World History we call the Enlightenment. Not so long ago, in a 9th grade class discussion, we arrived at the conclusion that the view of Locke was based on the teaching of people to be responsible citizens. We are children of Locke, not of Hobbes. If we do not remain so, we will, like Frenchmen in the terror, beg for a tyrant. Social Contract to the pessimistic Hobbes meant giving up freedom for safety. What a pity it will be if we are forced to be this way by a populace deliberately under taught by an elite with designs on power.
“High stakes testing violates human rights.”
Yes. Testing doesn’t just eat up learning time and provide monetizing personal information to people who’ve no right to it, it places children in an overly restrictive environment, not learning, answering questions for extended periods of time. What’s that called, Bush 2, enhanced interrogation, right?
YUP!
As someone who just recently went through the standardized testing process just four years ago, I agree wholeheartedly with this article. The fact that we given students a specific kind of curriculum to be tested on, as well as a significantly short time limit in which to complete each test section, is not applicable whatsoever to the varying kinds of students taking these tests. You could have a student who is highly intelligent but may need more time completing a question/set of questions than another student at the same intelligence level. It’s the same difference when it comes to differing speeds while reading. I believe a new system should be put into place that better reflects a student’s intelligence, ability, and personal worth.
I’m sorry for whoever said said that but I disagree completely, why? Right now there is no difference between the democracy in this country than the democracy in Cuba. First I graduated from the best navy academy in the world, because people here who don’t have gpa they’re lost in the ocean. If you don’t pass that test you lose those 5 years you’ve been training for, because the last results prove who the person is , because in the final tests you can’t cheat, in class you can. Last but not least THE WORST MOVEMENT ARE CHARTER SCHOOLS, kids who come in from Cuba in 5th grade know more than a student in 12th grade in a charter school. There are 2 types of dictatorships, the one that says and admits is a dictator ship and the one that hides themselves and doesn’t say they are dictators. I believe in the old education with no computers .