William Mathis is managing director of the National Education Policy Center and vice-chair of the Vermont Board of Education.
Mathis writes here about the inherent flaws of today’s standardized tests.
“They claim to measure “college and career readiness.” Yet, it takes no particular insight to know that being ready for the forestry program at the community college is not the same as astrophysics at MIT. Likewise, “career ready” means many different things depending upon whether you are a health care provider, a convenience store clerk, or a road foreman.
“The fundamental flaw is pretending that we can measure an educated person with one narrow set of tests. There is no one universal knowledge base for all colleges and careers. This mistake is fatal to the test-based reform theory.
“When the two test batteries (PARCC and SBAC) are put to the test, they don’t score very well. Princeton based Mathematica Policy Research compared PARCC test scores with freshman grade point average and found only 16 percent could be predicted (in the best case) by the math test and less than 1 percent by the English Language Arts score. The SBAC doesn’t have such a validity study but they say it “appears in their crystal ball.” (p.72 1). Since the future of schools and children are in the balance, this is no place for murky crystal balls…
“In the current latent traits fad, here’s how the tail has to wag:
“Knowledge can only have one line from easiest to hardest, children within a grade are equally distributed within and across all classrooms, and that all children learn the same things in the same way, in the same order and at the same time. As any parent of two or more children can tell you, that is not reality.
“Another fatal tail wagging is that no matter how important the item, if it doesn’t fit the latest test fad, it is tossed out. The result is that the test drifts off in space. This problem is made worse when politicians dangle money in front of test experts to do things with tests that cannot and should not be done, says Shavelson.
“If we redesigned our measures to address what our state constitutions and citizens tell us is important, we would concentrate on the skills that define success as a citizen, worker and human being. These which include clear and effective communication, creative and practical problem-solving, informed and integrative thinking, responsible and involved citizenship, and self-direction.
“This is not to say that standardized testing should be eliminated. It is the single uniform measure across schools. But the very standardized attributes that make them valuable cause harm to those things that are truly important for our children, and our communities.
“Since the “recommended” SBAC tests’ standards are currently set to fail about two-thirds of students, the data will wrongly and dishonestly provide fodder for school critics. In high scoring states, a mere half of students will be declared failures even though they would rank in the top 10 percent of the world. The test scores measure neither college nor careers nor success in life. They simply float free in monolithic space radiating glossy ignorance but as far as informing us about our schools, they are a cold, silent and misleading void.”
I have only one disagreement with Mathis’ keen analysis.
Given the pervasive misuse of standardized tests, our nation would benefit by having a moratorium on standardized testing of three to five years, during which time we might figure out how and when to use them, how to educate without them, and why test scores not the purpose of going to school.
Diane Ravitch's blog 
“The fundamental flaw is pretending that we can measure an educated person with one narrow set of tests.”
Well, close, but…. The fundamental flaw is pretending we can measure people period. Even if we’re trying to “measure” readiness for just one field or one area of knowledge, we can’t do it. You can’t measure qualities and people are never just quantities.
LikeLiked by 1 person
Hear, hear.
LikeLike
AMEN, Dienne. AGREE.
LikeLike
Bien dicho, Dienne.
On a side note, I hope to have the books by the weekend!
LikeLike
“Given the pervasive misuse of standardized tests, our nation would benefit by having a moratorium on standardized testing of three to five years….”
Why a moratorium? I remember when I was a student at U of C it was well known that nearly all the professors were softies for giving extensions, yet I never availed myself of the opportunity. I always figured, if I don’t know what I’m doing by the night before the paper is due, what am I going to suddenly learn/understand/realize in the next week or two or month or two or however long I could get an extension?
Similarly, what do we think will change in the next three to five years? Are we going to suddenly learn how to measure the unmeasurable? We’ve been trying at least my entire lifetime – and in the last decade or two we’ve been trying hard and harder. We’ve gotten nowhere – perhaps, just maybe, because there is nowhere to go? Maybe we should just write that paper now based on what we know and hand it in: there is no way to measure a human being, much less measure human beings against each other.
LikeLike
Like!
LikeLike
There is much to praise in this commentary, but there is also here a tendency to treat significant learning as content free, just a matter of acquiring skill sets in effective communication, problem-solving, etc.
We have a president who has some skills in communicating and these are appealing to his base. We have a president whose idea of problem solving is the destruction of democracy, and internationally, using nuclear weapons.
Trump’s apparent ignorance of the causes of the Holocaust, the legacy of lynching by the KKK, and the real consequences of nuclear war are among the many empty vessels–no there there–that allow him to pretend nothing matters except his own greatness. He has surrounded himself people who have smooth and fast talking “skill sets” and are just fine with saying: ” It is just wonderful to serve the Trump.”
It is all too easy to reduce education to a matter of learning skills and measurable competencies that strip content and issues from the curriculum and limit opportunities for students to practice making judgments that have consequences beyond self interest and easy measurement.
LikeLike
What a disastrous finding. Not that I’m one bit surprised. To think about the teachers and students that were forced to participate in this debacle, makes me ill.
LikeLike
I would like to see an in-depth study or two (or three) teasing out the possible characteristics of test takers within and among various groups. The only thing we keep spouting is that socioeconomic class and test scores are significantly correlated. That’s all fine and good. So let’s look at just high socioeconomic test takers and/or just test takers below the poverty line and/or lower middle class test takers… We can probably find lots of ways to cut the cake. We can’t say tests tell us nothing and then say high socioeconomic test takers do better because of the advantages they have. By the same token, why “broad brush” all test takers from below the poverty line? I know we have long discussions about characteristics (grit) that lead to success, but our argument that we learn nothing from standardized tests except probable socioeconomic class precludes such discussions. I AM NOT AN ADVOCATE OF HIGH STAKES TESTING! It serves the reform narrative to be able to label children and or schools as failing using tools that were never intended or designed to make such judgements. However, I would love to see if performance is influenced significantly by what the test taker believes or hopes they can achieve. As a former special ed teacher, I think I can make a stab at that question, but there is plenty of room for further investigation. If realistically a student expects to go to college or plans to be a checker at the local supermarket, what influence might that have on performance? Can we put together a profile of the “successful” test taker that spans the population of test takers and what does it tell us if we do? Saying they measure nothing is true, but that doesn’t mean there is no useful information to be gleaned from them with more research.
LikeLike
“Saying they measure nothing is true, but that doesn’t mean there is no useful information to be gleaned from them with more research.”
Since the data gleaned from the tests is totally corrupt, completely invalid, any usage of it to attempt to determine something (gleaned from them with more research) is ludicrous and risible. Nothing of value can be gleaned from invalid results other than some more invalidities.
Noel Wilson proved those invalidities in his never refuted nor rebutted dissertation*. A shorter discussion by Wilson, though, can be found in his review of the testing bible “Standards for Educational and Standardized Testing” found at: A Little Less than Valid: An Essay Review
http://edrev.asu.edu/index.php/ER/article/view/1372/43
*for a summary of that dissertation, please see below.
LikeLike
You and I have had this discussion before, Duane. I should have been clearer in my comment. Gleaning info from these instruments does not mean taking measurements, and getting info from them does not mean using them to make fraudulent leaps of logic. I was in the “business” of creating a picture of a particular student based on a wide range of information: anecdotal, observational, work product, assessments,… I am not ready to concede that there is no useful info for creating such pictures of larger classes of students using standardized tests. Are the results totally screwed/skewed by the prejudices of the test makers or is there information that can be combined with other sources that might give us some insights into different sub populations? I don’t know, but I wonder. Why is it that I could predict with some accuracy the way my students would perform on assessments? What about the students and what about those assessments let me form a fairly accurate picture of their performance before they took a test? Is there a place for standardized tests in high stake decision making? NO, NO, NO! We never used testing alone in making decisions about individual students; testing was used to confirm and/or question what we saw in the classroom as well as in the wider environment.
LikeLike
“Why is it that I could predict with some accuracy the way my students would perform on assessments?”
Because as their teacher you are far more knowledgeable about them and those standardized assessments (I’m not talking of specific diagnostic tests) offer little to no more insight than you already have. Trust yourself and your professional being. Do not succumb to the supposed and false “authority” that are standardized tests.
LikeLike
What makes standardized tests that are used for diagnostic tests any different than any other standardized test?
LikeLike
The difference in diagnostic tests is that there are no correct answers. They are batteries of questions and depending upon the responses a diagnosis may be made. The diagnostic tests are not designed to sort and separate students into a bell curve of ranking.
Does that help?
LikeLike
Yes, your explanation helps although it is not true of all the tests that are used diagnostically. We used tests with “correct” answers diagnostically, too. The difference was in what we used them for. I am interested in a similar diagnostic process being applied to all the standardized instruments that have been used to rank and sort. Key to the whole process is, as you say in the case of strictly diagnostic instruments, what is done with the data. There appear to be patterns which I would like to explore more diagnostically. Designing such studies are beyond me. I agree with you that the use of standardized tests to make high stakes decisions is wrong. No one has designed a test that should be allowed to decide a student’s worth or future course in isolation.
LikeLike
cx; used for diagnostic purposes
LikeLike
Already in 1999, the former AERA president Alan Schoenfeld has suggested a moratorium on conventional testing because of the many unsolved problems of vality and statistic.
Conventional tests (classical and modern test theory) had been promoted in the US by the engineering professor Thurstone of the University of Chicago. Engineers are only concerned is with measurement error and how to minimize it: “The main purpose of Classical Test Theory within psychometric testing is to recognise and develop the reliability of psychological tests and assessment…” Why? Because they know their object of measurement well and do not be concerned wirh validity.
Yet psychology and education struggle with the meaning of their measurement objectives (what is “personality”?) and are tehrefore concerned mainly with validity, not reliability: Do we really measure whet we think we do? Do we know what we what we want to measure?
For such questions, neither classical test theory nor its modern variants have an answer.
Actually it is hard to believe that almost all psychologist and educators who do measurement have baught into this totally inadequate measurement practice.
Yet there are alternatives possible. Old time experimental psychology (Donders, Brunswik etc) have formuated exciting alternatives — which have hardly ever been noticed. On this basis I have developed a new measurement methodology – Experimental Questionnaire (EQ) – with which, for example, we can now measure moral competence in a valid way: The Moral Competence Test (MCT). This methodology can be easily applied on other subjects, too, in order to cope with the validy problem, because it is strictly based on psychological reseasrch and sound theory instead of on doubious statistical model with not validity.
The only drawback is that the application of this new methods requires thorough knowledge of current psychological and educational research on the concept which is to be measured, and it wil cost the test industry a lof of money because they will have to through away all their tests.
Let’s go ahead.
GEorg
LikeLike
“On this basis I have developed a new measurement methodology – Experimental Questionnaire (EQ) – with which, for example, we can now measure moral competence in a valid way: The Moral Competence Test (MCT).”
Oh take your sales pitch elsewhere. You can’t measure moral competence. Period. First you’d have to define it and I challenge you to do even that much. But, much more challengingly, you’d have to come up with a standard unit of moral competence. Good luck with that.
LikeLike
Incidentally, when making a sales pitch, proofreading is your friend.
LikeLike
“On this basis I have developed a new measurement methodology – Experimental Questionnaire (EQ) – with which, for example, we can now measure moral competence in a valid way: The Moral Competence Test (MCT). This methodology can be easily applied on other subjects, too, in order to cope with the validy problem, because it is strictly based on psychological reseasrch and sound theory instead of on doubious statistical model with not validity.”
No Georg, you haven’t even begun to address any of Wilson’s concerns. I guarantee that your MCT suffers all the onto-epistemological falsehoods and errors as those prior tests that you decry.
One cannot measure the inmeasurable.
Your test may assess, judge, evaluate, but it doesn’t measure anything.
LikeLike
Ok, Senor Swacker, time for another Wilson rant (for those many new readers on this blog {& I’ve noticed a lot of new commenters here lately}–great, & all of us “oldies but goodies”: keep spreading the word that this blog is a must read).
LikeLike
Done!
LikeLike
Oh, & I meant to say (& always ask a public school educator, who has known for YEARS, w/o having to read any studies, that there are “inherent flaws”): these tests TELL US
NOTHING…NOTHING. Nada. Zip. Zero. 0.
Except how much real, valuable teaching/learning/recess (see a later post) time is WASTED on endless test prepping.
Except how many taxpayer (you & I) $$$$$$ are WASTED on test prep materials & tests.
Except how much media center/library/computer lab time is WASTED for pre-empting these areas for TEST TAKING.
Except how young children & special education students who may, first, be lacking in eye-hand coordination skills, suffer confusion, embarrassment & loss of self-esteem when forced to test on computers (or even filling in Scantron bubbles).
And there’s a whole laundry list more.
Aside from the fact that these are NOT “standardized” tests. No quality control, accountability, unqualified scorers hired from Craig’s list (& even before there was a Craig–simply read Todd Farley’s 2009 {!} book, “Making the Grades: My Misadventures in the Standardized Testing Industry”–same incompetents being hired today.)
LikeLike
Like!
LikeLike
I’m so angry I didn’t make my last sentence complete: recycled questions (using, again, even the poorly written ones) and questions having no right answer, more than one right answer, and on & on, ad nauseum.
LikeLike
You guys sure aren’t making it easy for John Fallon @ Pearson to make his numbers.
Who will you kick once Pearson is dead and gone?
LikeLike
Do you really want a list?
LikeLike
Pearson is in fact shrinking. Time to start exposing the rest of the list: McGraw Hill, Measurement Incorporated, AIR, Measured Progress, DRC…
LikeLike
I’d very much like to kick McGraw Hill once Pearson is down. Oh, and College Board too.
LikeLike
Diane,
Have you ever read S.J. Gould’s book, “The Mismeasure of Man”? I think it’s his most brilliant work, and it aims at the heart of the beast. Although Gould addresses ‘intelligence’ tests, in particular (he’s been dead for a few years, now), such ‘instruments’ as the SAT are simple outgrowths of the floundering ‘IQ’ tests. (note: SAT stands for scholastic ‘Aptitude’).
Today’s tests suffer from the same circular logic. They assume we not only know that what we are measuring is a physical reality, but that we also know which people have that quality. Indeed, when testing companies include unscored questions in their tests, the entire purpose is to make sure that the ‘right’ people get the correct answers before they foist those questions on the general population. This shows not only intellectual dishonesty but also the poverty of our understanding of the vast range of human capacity needed to survive in the natural world.
I think the word ‘education’ says, clearly, what we (teachers, gurus, older humans) ought to do. We need to ‘draw out’ each student in a way that supports what is within that particular student. Only in this way can we help our children into an uncertain future.
Each of us, of course, will ‘draw’ in a somewhat different way, and so the student will be stretched and expanded as a result.
LikeLike
Also, I would recommend Gould’s Full House, a response to a book called The Bell Curve. There Gould pointed out that statistical measures of baseball suggest the futility of trying to measure mentality.
Historians who have tried to understand the public mind have generally fallen quite flat in their search for mentalite or zeitgeist. By their own honest admission, their thoughts are but probes into the deep space of society’s thinking. It is time for honest psychologists to admit the same thing. The best idea is but an idea. We should voice these ideas, but we should never claim a numerical edge over those who disagree with us.
Furthermore, in the apparent absence of our friend Duane, someone should point out that Wilson’s thesis has never been refuted.
LikeLike
Just arrived to the party a tad late. See below for why. Too nice an evening to be inside.
LikeLike
Do you know the worst standardized test? The CRT4. It is used by private schools to prove their students are better educated. If you ask them for information on how they gleaned public school norms without giving public school students the test, they have no real answer. The test is so proprietary they share little information. Private schools use it to feel superior. I wouldn’t be surprised if data collected from this non-profit is used to promote school choice. Urrg!
LikeLike
“The test is so proprietary they share little information.”
Because if they did the test would be laughed off the face of this continent.
LikeLike
“They claim to measure “college and career readiness.”
Hmmm. . . I wonder what the standard unit of measurement for “college and career readiness” is???? Where can we find an exemplar of that standard unit to base or calibrate our supposed measuring device on???
Oh. . . What??? There is no standard unit of “college and career readiness”???
How the hell does one measure it then???
LikeLike
Just got back from sitting out in the countryside with a friend “Tomato Tom” on his back “yard” looking up through a 50 acre field to a tree line up the hill waiting for the deer to come browsing through. And they didn’t disappoint. Beautiful 80 degree evening!
So I arrive at this discussion a little late and will honor RBMTK’s request for a summary of Wilson’s work:
“Educational Standards and the Problem of Error” found at:
http://epaa.asu.edu/ojs/article/view/577/700
Brief outline of Wilson’s “Educational Standards and the Problem of Error” and some comments of mine.
A description of a quality can only be partially quantified. Quantity is almost always a very small aspect of quality. It is illogical to judge/assess a whole category only by a part of the whole. The assessment is, by definition, lacking in the sense that “assessments are always of multidimensional qualities. To quantify them as unidimensional quantities (numbers or grades) is to perpetuate a fundamental logical error” (per Wilson). The teaching and learning process falls in the logical realm of aesthetics/qualities of human interactions. In attempting to quantify educational standards and standardized testing the descriptive information about said interactions is inadequate, insufficient and inferior to the point of invalidity and unacceptability.
A major epistemological mistake is that we attach, with great importance, the “score” of the student, not only onto the student but also, by extension, the teacher, school and district. Any description of a testing event is only a description of an interaction, that of the student and the testing device at a given time and place. The only correct logical thing that we can attempt to do is to describe that interaction (how accurately or not is a whole other story). That description cannot, by logical thought, be “assigned/attached” to the student as it cannot be a description of the student but the interaction. And this error is probably one of the most egregious “errors” that occur with standardized testing (and even the “grading” of students by a teacher).
Wilson identifies four “frames of reference” each with distinct assumptions (epistemological basis) about the assessment process from which the “assessor” views the interactions of the teaching and learning process: the Judge (think college professor who “knows” the students capabilities and grades them accordingly), the General Frame-think standardized testing that claims to have a “scientific” basis, the Specific Frame-think of learning by objective like computer based learning, getting a correct answer before moving on to the next screen, and the Responsive Frame-think of an apprenticeship in a trade or a medical residency program where the learner interacts with the “teacher” with constant feedback. Each category has its own sources of error and more error in the process is caused when the assessor confuses and conflates the categories.
Wilson elucidates the notion of “error”: “Error is predicated on a notion of perfection; to allocate error is to imply what is without error; to know error it is necessary to determine what is true. And what is true is determined by what we define as true, theoretically by the assumptions of our epistemology, practically by the events and non-events, the discourses and silences, the world of surfaces and their interactions and interpretations; in short, the practices that permeate the field. . . Error is the uncertainty dimension of the statement; error is the band within which chaos reigns, in which anything can happen. Error comprises all of those eventful circumstances which make the assessment statement less than perfectly precise, the measure less than perfectly accurate, the rank order less than perfectly stable, the standard and its measurement less than absolute, and the communication of its truth less than impeccable.”
In other words all the logical errors involved in the process render any conclusions invalid.
The test makers/psychometricians, through all sorts of mathematical machinations attempt to “prove” that these tests (based on standards) are valid-errorless or supposedly at least with minimal error [they aren’t]. Wilson turns the concept of validity on its head and focuses on just how invalid the machinations and the test and results are. He is an advocate for the test taker not the test maker. In doing so he identifies thirteen sources of “error”, any one of which renders the test making/giving/disseminating of results invalid. And a basic logical premise is that once something is shown to be invalid it is just that, invalid, and no amount of “fudging” by the psychometricians/test makers can alleviate that invalidity.
Having shown the invalidity, and therefore the unreliability, of the whole process Wilson concludes, rightly so, that any result/information gleaned from the process is “vain and illusory”. In other words start with an invalidity, end with an invalidity (except by sheer chance every once in a while, like a blind and anosmic squirrel who finds the occasional acorn, a result may be “true”) or to put in more mundane terms crap in-crap out.
And so what does this all mean? I’ll let Wilson have the second to last word: “So what does a test measure in our world? It measures what the person with the power to pay for the test says it measures. And the person who sets the test will name the test what the person who pays for the test wants the test to be named.”
In other words it attempts to measure “’something’ and we can specify some of the ‘errors’ in that ‘something’ but still don’t know [precisely] what the ‘something’ is.” The whole process harms many students as the social rewards for some are not available to others who “don’t make the grade (sic)” Should American public education have the function of sorting and separating students so that some may receive greater benefits than others, especially considering that the sorting and separating devices, educational standards and standardized testing, are so flawed not only in concept but in execution?
My answer is NO!!!!!
One final note with Wilson channeling Foucault and his concept of subjectivization:
“So the mark [grade/test score] becomes part of the story about yourself and with sufficient repetitions becomes true: true because those who know, those in authority, say it is true; true because the society in which you live legitimates this authority; true because your cultural habitus makes it difficult for you to perceive, conceive and integrate those aspects of your experience that contradict the story; true because in acting out your story, which now includes the mark and its meaning, the social truth that created it is confirmed; true because if your mark is high you are consistently rewarded, so that your voice becomes a voice of authority in the power-knowledge discourses that reproduce the structure that helped to produce you; true because if your mark is low your voice becomes muted and confirms your lower position in the social hierarchy; true finally because that success or failure confirms that mark that implicitly predicted the now self-evident consequences. And so the circle is complete.”
In other words students “internalize” what those “marks” (grades/test scores) mean, and since the vast majority of the students have not developed the mental skills to counteract what the “authorities” say, they accept as “natural and normal” that “story/description” of them. Although paradoxical in a sense, the “I’m an “A” student” is almost as harmful as “I’m an ‘F’ student” in hindering students becoming independent, critical and free thinkers. And having independent, critical and free thinkers is a threat to the current socio-economic structure of society.
LikeLike
Gracias, Senor S!
(& hope you were able to fend off the mosquitoes!)
LikeLiked by 1 person
One more thought:
Whatever is measured counts
Whatever counts is measured
And counting whatever measures
Is measuring whatever counts
SomeDam Poet.
The most misleading concept/term in education is “measuring student achievement” or “measuring student learning”. The concept has been misleading educators into deluding themselves that the teaching and learning process can be analyzed/assessed using “scientific” methods which are actually pseudo-scientific at best and at worst a complete bastardization of rationo-logical thinking and language usage.
There never has been and never will be any “measuring” of the teaching and learning process and what each individual student learns in their schooling. There is and always has been assessing, evaluating, judging of what students learn but never a true “measuring” of it.
But, but, but, you’re trying to tell me that the supposedly august and venerable APA, AERA and/or the NCME have been wrong for more than the last 50 years, disseminating falsehoods and chimeras??
Who are you to question the authorities in testing???
Yes, they have been wrong and I (and many others, Wilson, Hoffman etc. . . ) question those authorities and challenge them (or any of you other advocates of the malpractices that are standards and testing) to answer to the following onto-epistemological analysis:
The TESTS MEASURE NOTHING, quite literally when you realize what is actually happening with them. Richard Phelps, a staunch standardized test proponent (he has written at least two books defending the standardized testing malpractices) in the introduction to “Correcting Fallacies About Educational and Psychological Testing” unwittingly lets the cat out of the bag with this statement:
“Physical tests, such as those conducted by engineers, can be standardized, of course [why of course of course], but in this volume , we focus on the measurement of latent (i.e., nonobservable) mental, and not physical, traits.” [my addition]
Notice how he is trying to assert by proximity that educational standardized testing and the testing done by engineers are basically the same, in other words a “truly scientific endeavor”. The same by proximity is not a good rhetorical/debating technique.
Since there is no agreement on a standard unit of learning, there is no exemplar of that standard unit and there is no measuring device calibrated against said non-existent standard unit, how is it possible to “measure the nonobservable”?
THE TESTS MEASURE NOTHING for how is it possible to “measure” the nonobservable with a non-existing measuring device that is not calibrated against a non-existing standard unit of learning?????
PURE LOGICAL INSANITY!
The basic fallacy of this is the confusing and conflating metrological (metrology is the scientific study of measurement) measuring and measuring that connotes assessing, evaluating and judging. The two meanings are not the same and confusing and conflating them is a very easy way to make it appear that standards and standardized testing are “scientific endeavors”-objective and not subjective like assessing, evaluating and judging.
That supposedly objective results are used to justify discrimination against many students for their life circumstances and inherent intellectual traits.
C’mon test supporters, have at the analysis, poke holes in it, tell me where I’m wrong!
I’m expecting that I’ll still be hearing the crickets and cicadas of tinnitus instead of reading any rebuttal or refutation.
Because there is no rebuttal/refutation!
LikeLike
Thanks Duane. We have been seeing a lot of deer movement during our rather wet summer. All the young still have their spots and are running around now.
LikeLiked by 1 person
On another matter, I wonder how you would react to the testing experienced by my old friend and professor who got his degree from King’s College back in the forties. He took two tests that he recalls, one to be accepted into the graduate program in biology and one to receive his degree.
The first consisted of sitting for a four to five hour essay describing the unity described by a multitude of natural history artifacts. Bones, skins, and preserved specimen were placed on a table, and the young teenagers attempted to show that they were well enough versed in the significance of them to be allowed to study.
The next time he took a test, it was to see whether he would get one of five levels of recommendation. It was similar. What do you think of that approach to testing?
LikeLiked by 1 person
That sort of assessment will be decried by the standards and testing folks as biased and subjective. Wilson delves into that in his dissertation mentioned in this thread and I briefly touch on it in my book.
I think it can be a valid way of assessing a student’s knowledge however these days the process would probably scare the shit out of the candidate for not ever having done that sort of thing.
I proclaim no answers to these types of questions. I show the inherent falsehoods and errors which should lead us to reject the standards and testing regime. As I say in the Afterword in the book:
A tactic of administrators or any powers that be to silence those bold enough to critique their policies and practices, even after agreeing with one’s critique, is “Well, you’ve criticized what we are doing but “What is your solution?” usually said with such tone and emphasis as if they have now trapped the perpetrator in a debate dilemma. The administrator knows that it is impossible to come up with a feasible solution to your critiques in the minute or two they allot you to do so, solving his/her problem of the critical thinker in their employ. He/She walks away smug in his/her confidence that he/she won that verbal battle. And you’re left standing there thinking “What a smug ass bastard!”
It takes an immense amount of ego, hubris and gall to think that one person can solve long standing, seemingly intractable structural problems in the public education realm especially on such short notice. To attempt to do so guarantees failure. Not only that but who am I to propose solutions for everyone else? Our society doesn’t work that way. So I offer no specific answers but I do offer some general guidelines in struggling to lessen the many injustices that current educational malpractices entail:
• Correctly identify malpractices that hinder the teaching and learning process and that cause harm to or do injustice to students. (see just a few identified above).
• Immediately reject those malpractices, cease doing them as soon as is practically possible.
• Maintain a “fidelity to truth” attitude in identifying those malpractices and instituting new practices.
• Focus on inputs and resources. Are they adequate to provide that all children have access to a learning environment in which they can learn to “savor the right to life, liberty, the pursuit of happiness, and the fruits of their own industry”?
• Involve all, interested community members, parents, students, teachers, aides, other support personnel, administrators and the school board in revising and formulating new policies and practices so that, paraphrasing the disembodied voice from the movie “Field of Dreams”:
IF WE PROVIDE IT, THEY WILL COME!
It being the proper resources implemented with a fidelity to truth attitude.
They being results in line with the fundamental purpose(s) of public education–to promote the welfare of the individual so that each person may savor the right to life, liberty, the pursuit of happiness, and the fruits of their own industry.
LikeLike
Thanks for your response. I really like the idea of ruling out practices that are obviously malpractices. If we cannot do that, and there is indication that we are currently unable to do that, then the process you describe is quite impossible. Most of us around this blog seem to think there are agendas hidden from society that drive our inability to come to rationally reached conclusions that would become policies. I add my own belief to those. Policies are driven by political and social implications for the players in a game that seems bigger than the classroom. To quote Diane: Sad.
LikeLike
Most teachers claim that there is too much testing. (I am not a teacher, but I am inclined to agree with the majority). I have never heard a teacher nor an administrator claim that there was not enough testing.
Nevertheless, I am struggling to find out, how the “product” (education) is to be measured and quantified. How is the public, to be informed of the efficacy of the schools which are operated with their tax dollars? What kind of metric is to be used?
Isn’t it important to know, what you don’t know?
LikeLike
How should we quantify your worth, Charles? Your bank account? Your weight? Height?
LikeLike
First, Chas one has to define the “product”. What is the “product” of the teaching and learning process? Define that for us and we can maybe begin to have a conversation about how to assess that “product”. And, yes, I have a bit of production scheduling and materials management background in which knowing what the product is is the first step to making said product. So, again, lets start with you defining the product of the teaching and learning process.
Now, how then do we assess the efficacy? Well, again without knowing what the product is, we cannot begin to assess the efficacy of the process. And as far as what “metric”, again without knowing the product, how can we figure out a metric?
Help me out Chas, what is that product?
LikeLike
We used to leave decisions like that up to the teacher. Not until the last two decades began to try to think of education as a product have we eroded teacher authority with testing.
When a teacher becomes a mere vehicle for passing a test, the moral authority and the intellectual authority are undermined. People who teach become transmitters of lists, not authorities in their professions.
The public may be assured their children are getting a good education when children come home talking about history and math. Unfortunately for those who want metrics, the only real things we can measure are the number of books in the library and the number of days students come to school. All that other stuff is smoke and mirrors.
LikeLike
For those worried (!) about Pear$on: 3,000 employees were recently laid off.
But you can bet the CEO & upper execs are making do w/their big $$$$$!
Even as their K-12 tests are CCRAP, they’ve cornered the GED & the student teacher evaluation markets.
Summer’s almost over; pick a Pear$on campus near you & have a protest.
LikeLike
Reblogged this on patthaleblog and commented:
The best part:
“If we redesigned our measures to address what our state constitutions and citizens tell us is important, we would concentrate on the skills that define success as a citizen, worker and human being. These which include clear and effective communication, creative and practical problem-solving, informed and integrative thinking, responsible and involved citizenship, and self-direction.”
LikeLike