Phil Cullen is a blogger in Australia, whose blog “The Treehorn Express” eloquently savages national standards and testing.
Cullen discovered our reader Duane Swacker, who teaches in Missouri. Duane regularly lectures us on Noel Wilson’s devastating critique of standardized testing.
I am happy to point out that Phil Cullen discovered Duane Swacker and introduced Noel Wilson’s thinking to many Australian educators. (I take credit for match-making.)
Phil Cullen wrote:
Dear NAPLAN victim,
When you receive your score or mark or judgement from your NAPLAN test, it becomes a part of your life…part of what you are…..part of society’s view of you….a serious part……a very serious part… It can be a proper bastard.
You have been mushroomed, as have your parents and there is no way out of it. You are branded for life.
Notable Missouri commentator, Duane Swacker, in his review of Noel Wilson’s dissertation on “EDUCATIONAL STANDARDS AND THE PROBLEM OF ERROR” makes some very telling points about the branding of children with a score and the self-fulfilling prophecy of such mutilation. If you have read it before, please read it again. It’s very, very important. It’s serious.
“…..This mark becomes part of the story about yourself and with sufficient repetitions becomes true: true because those who know, those in authority, say it is true; true because the society in which you live legitimates this authority; true because your cultural habitus makes it difficult for you to perceive, conceive and integrate those aspects of your experience that contradict the story; true because in acting out your story, which now includes the mark and its meaning, the social truth that created it is confirmed; true because if your mark is high you are consistently rewarded, so that your voice becomes a voice of authority in the power-knowledge discourses that reproduce the structure that helped to produce you; true because if your mark is low your voice becomes muted and confirms your lower position in the social hierarchy; true finally because that success or failure confirms that mark the implicity predicted the now self-evident consequences. And so the circle is complete.” [Noel Wilson: “Educational Standards and the Problem of Error. http://epaa.asu.edu/ojs/article/view/577/700 ]
*Earlier in his article, Swacker points out the dangers in attempting to quantify aspects of a person’s personal quality. “It is illogical” he says “to judge/assess a whole category only by a part of the whole….In attempting to quantify educational standards by standardised testing the descriptive information about said interactions is inadequate, insufficient and inferior to the point of invalidity and unacceptability.”
* An even greater epistemological mistake is to extend the scores not only to the pupils, but also to the teacher, the school and the district. “This error is probably one of the most egregious ‘errors’ that occur with standardised testing.” adds Swacker.
Swacker’s full article challenges the thoughts of Naplanic testucators in a more serious way than is usual……The issues highlight the importance of Australia’s need to THINK seriously about the nature of schooling in this country.
DUANE SWACKER REVIEWS NOEL WILSON
Brief outline of Wilson’s “Educational Standards and the Problem of Error” and some comments of mine. (updated 6/24/13 per Wilson email)
1. A description of a quality can only be partially quantified. Quantity is almost always a very small aspect of quality. It is illogical to judge/assess a whole category only by a part of the whole. The assessment is, by definition, lacking in the sense that “assessments are always of multidimensional qualities. To quantify them as unidimensional quantities (numbers or grades) is to perpetuate a fundamental logical error” (per Wilson). The teaching and learning process falls in the logical realm of aesthetics/qualities of human interactions. In attempting to quantify educational standards and standardized testing the descriptive information about said interactions is inadequate, insufficient and inferior to the point of invalidity and unacceptability.
2. A major epistemological mistake is that we attach, with great importance, the “score” of the student, not only onto the student but also, by extension, the teacher, school and district. Any description of a testing event is only a description of an interaction, that of the student and the testing device at a given time and place. The only correct logical thing that we can attempt to do is to describe that interaction (how accurately or not is a whole other story). That description cannot, by logical thought, be “assigned/attached” to the student as it cannot be a description of the student but the interaction. And this error is probably one of the most egregious “errors” that occur with standardized testing (and even the “grading” of students by a teacher).
3. Wilson identifies four “frames of reference” each with distinct assumptions (epistemological basis) about the assessment process from which the “assessor” views the interactions of the teaching and learning process: the Judge (think college professor who “knows” the students capabilities and grades them accordingly), the General Frame-think standardized testing that claims to have a “scientific” basis, the Specific Frame-think of learning by objective like computer based learning, getting a correct answer before moving on to the next screen, and the Responsive Frame-think of an apprenticeship in a trade or a medical residency program where the learner interacts with the “teacher” with constant feedback. Each category has its own sources of error and more error in the process is caused when the assessor confuses and conflates the categories.
4. Wilson elucidates the notion of “error”: “Error is predicated on a notion of perfection; to allocate error is to imply what is without error; to know error it is necessary to determine what is true. And what is true is determined by what we define as true, theoretically by the assumptions of our epistemology, practically by the events and non-events, the discourses and silences, the world of surfaces and their interactions and interpretations; in short, the practices that permeate the field. . . Error is the uncertainty dimension of the statement; error is the band within which chaos reigns, in which anything can happen. Error comprises all of those eventful circumstances which make the assessment statement less than perfectly precise, the measure less than perfectly accurate, the rank order less than perfectly stable, the standard and its measurement less than absolute, and the communication of its truth less than impeccable.”
In other words all the logical errors involved in the process render any conclusions invalid.
5. The test makers/psychometricians, through all sorts of mathematical machinations attempt to “prove” that these tests (based on standards) are valid-errorless or supposedly at least with minimal error [they aren’t]. Wilson turns the concept of validity on its head and focuses on just how invalid the machinations and the test and results are. He is an advocate for the test taker not the test maker. In doing so he identifies thirteen sources of “error”, any one of which renders the test making/giving/disseminating of results invalid. And a basic logical premise is that once something is shown to be invalid it is just that, invalid, and no amount of “fudging” by the psychometricians/test makers can alleviate that invalidity.
6. Having shown the invalidity, and therefore the unreliability, of the whole process Wilson concludes, rightly so, that any result/information gleaned from the process is “vain and illusory”. In other words start with an invalidity, end with an invalidity (except by sheer chance every once in a while, like a blind and anosmic squirrel who finds the occasional acorn, a result may be “true”) or to put in more mundane terms crap in-crap out.
7. And so what does this all mean? I’ll let Wilson have the second to last word: “So what does a test measure in our world? It measures what the person with the power to pay for the test says it measures. And the person who sets the test will name the test what the person who pays for the test wants the test to be named.”
In other words it attempts to measure “’something’ and we can specify some of the ‘errors’ in that ‘something’ but still don’t know [precisely] what the ‘something’ is.” The whole process harms many students as the social rewards for some are not available to others who “don’t make the grade (sic)” Should American public education have the function of sorting and separating students so that some may receive greater benefits than others, especially considering that the sorting and separating devices, educational standards and standardized testing, are so flawed not only in concept but in execution?
My answer is NO!!!!!
One final note with Wilson channeling Foucault and his concept of subjectivization:
“So the mark [grade/test score] becomes part of the story about yourself and with sufficient repetitions becomes true: true because those who know, those in authority, say it is true; true because the society in which you live legitimates this authority; true because your cultural habitus makes it difficult for you to perceive, conceive and integrate those aspects of your experience that contradict the story; true because in acting out your story, which now includes the mark and its meaning, the social truth that created it is confirmed; true because if your mark is high you are consistently rewarded, so that your voice becomes a voice of authority in the power-knowledge discourses that reproduce the structure that helped to produce you; true because if your mark is low your voice becomes muted and confirms your lower position in the social hierarchy; true finally because that success or failure confirms that mark that implicitly predicted the now self-evident consequences. And so the circle is complete.”
Phil ….still caring about kids.
{Formerly: Treehorn Institute for Real Education}
Again
I believe Duane has retired.
You are correct, Abigail!
While I wouldn’t mind subbing, my stupid body won’t allow it. And that’s being generous, you should have seen what I called it in an email to my golfing buddies letting them know I couldn’t play golf yesterday-jeez can’t even play nine with a cart, yes it’s more than stupid.
ah, but your brain seems to be working very well 🙂
Thanks for the kind words, Ciedie!! Te amo.
But you can still camp, can’t you? (I refuse to ask you about hunting.)
Yes, Máté, I can still camp. I plan on camping my way out west in October to go to the NPE conference. Camp while I’m there and then camp on the way back.
And fish. Just went with my son on Saturday. Caught enough to make fish tacos, sides of refried black beans and red rice and some tamarind Jarritos to drink! Never have been that much of a hunter. I prefer the piscatorial pleasures.
It takes a Missourian to introduce an Australian’s work to Australia.
Strange world indeed, FLERP!!
Yay, Duane! Much deserved kudos.
I always did think that there was something particularly astute about Missourians. Especially those from St. Louis.
(Although, those of you still living in Missouri really need to do something about your governor, and most of your legislators.)
You are quite correc,t Zorba, in your parenthetical thought! A former student of mine, who claims that I changed his thinking from conservative to liberal, which I deny, he did it himself, has been an intern first, then paid aide to one of the Democratic representatives. He’s getting out because he can’t make a living being an aide. I think he’s seen just how smarmy Jeff City is and doesn’t like what he sees. We’re trying to get together over a beer soon (I’ll have one and even that one will “hurt” me the next day-oh for the olde days, eh!) and I’m hoping to hear that he has decided to go to law school like he has been talking about. He has a keen sense of fairness, equity and justice and I hope he can develop that even further.
Keep preachin’ the word, brother!
Like!
Wilson’s work through Duane reveals that labels can be self self fulfilling prophecies. High stakes testing enhances early ranking and sorting of young people that can limit opportunities for those that “don’t make the grade.” This is harmful policy. I have been a keen observer of students over the years. Working with ELLs I have worked with many under educated young people. Many of them needed time to catch up on academics and English. I have learned that passing early judgment on students can be harmful to this population as it can deny them access to more challenging academics. I am not against students getting the services they need, but early judgments about students’ potential work against poor and foreign young people that may be late bloomers due to their circumstances.
Like!!
Duane: You’ll understand this. There is also a difference in the cultural perceptions of labels, and this may have devastating consequences for some young people whose cultural background refuses to accept “labels.” Whereas in my school district, it was the middle class parents that were clamoring for the “learning disabled” designation so their little “snowflakes” could get extra time on bubble tests!
You are so right, retired teacher. All 3 of my sons suffered from this pubsch labeling in the ’90’s/’00,s. So different from the public-ed environment of the ’50’s-’60’s (when I was a kid): hi-IQ was picked out easily then, & not obfuscated by behavior issues; boys were expected to be obstreperous & that did not interfere w/assessing their aptitude.
Truer words never spoken.
My eldest’s saving grace– despite having been early-flagged as math-challenged– was that he was preturnaturally articulate. Regardess of the culture’s STEM-obsession, we listen to kids who can speak well. Despite his his SpEd IEP & supposed math-challenges (per test results), he was capable of explaining advanced math concepts to the layman [despite inability to work a complex math problem to its conclusion.] That ability hooked his middle-school math-mentor, who went the extra mile to ensure his test success. He had mental & physical challenges, but the hisch SpEd program (incl self-contained classes, which allowed him to advance even post-hospitalizations) kept him current w/his class & he went on to succeed in college.
Another example: I knew he had the ear & IQ [& genes] to be a linguist, but mid-sch for-lang courses in his day were all about reading& writing, which slowed his learning due to his LD’s. He would have excelled in the sort of conversational course that is available in some hischs today…
My middle son showed reading prowess early on, & OK math progress, so teachers did not notice what I saw: his grades in his favorite content subjects (science & social-studies) steadily nose-dived between 6th & 10thgrades. He was the classic mid-achiever who was ignored by teachers because he was neither hi- nor lo-achieving. Thank god our district hisch had a ’70’s-era alternative ‘school-w/n-a-school’ which targeted non-conformist low-achievers & I convinced him to sign on jr yr. It made all the difference. He went on to college, & continues as an auto-didact.
My youngest was labeled as SpEd [ADD] from the start, & repeated K w/an IEP; he has always had to fight feeling ‘dumb’ compared to peers despite advanced/ acknowledged superiority in music/ rhythm throughout K12. He like eldest bro benefited from small self-contained classes in hs, & continued on sched w/ peers to get a BA, & today has students lined up for his piano lessons.
My 3 kids varied in IQ from mid to high, but all were talented musicians & computer-techies from a very young age. And all 3 were labeled ‘not as good as’ math/eng academic peers from the moment they started K-12. They– & we as parents — had to fight that label, & find them venues w/n & extracurricular pubsch K12 where they could begin to understand their place in the greater world.
You did very right by your children bethree5! Many kudos to you and your children.
The services your children received were due to having well funded public schools. Imagine these children today, rejected by charters, and ending up in under resourced public schools. My daughter has emotional issues, and I thank God that the public school worked with us to enable her to graduate as she fell apart in high school.
Making an impact a half-a-world away. Spectacular.
Sadly, at the same time it says something about just how widespread the standards and testing regime has spread. GERM anyone??
I shudder at the thought of me being a child today. For sure, the billionaires, like Betsy (the devil’s bullet) DeVos and Bill G(r)ates, that worship at the greed-is-great altar of for-profit, flawed, secretive high-stakes testing, would have branded me as a total loser in the global data base for all to see if they were willing to buy that info.
As a child, I had serious health issues and dyslexia before they knew what dyslexia was and was branded back then, by a limited system without an unlimited internet to post that info on-line, as too retarded to learn to read and write. I was also a horrible test taker and I still am. Since there was no global database back then, there was no way for the world to add my name to the list of losers destined to stay in poverty and become a future prison inmate, and I went on to defy the alleged experts and ended up going to college on the GI Bill, earning a BA in journalism, a teaching credential, and an MFA in writing in addition to writing and publishing four award-winning books.
And that labeling, as in what happened to you, Lloyd, is one of the most egregious errors pointed out by Wilson in his never refuted nor rebutted dissertation “Educational Standards and the Problem of Error” found at http://epaa.asu.edu/ojs/article/view/577/700
Both Edison and Einstein were considered “damaged goods.” You’re in good company, Lloyd!
I so relate, Lloyd. I am one of 4 siblings whose parents were an Ivy-League BA (Mom) married to a highly-creative tradesman w/an 8th-gr education (Dad). Our family biz [initiated by Dad & funded by Mom] moved from auto-mechanics to house-building to land-development. Dad had dyslexia– which we did not figure out for decades.
I was fortunate to sail thro K12 & college. My youngest bro was equally-unencumbered by dyslexia, & made a career in culinary arts.
But the next 2 siblings had severe dyslexia– tho we did not figure that out for long, because dyslexia was not even known/ named when my next-youngest bro was coming thro K12. My Mom fought pubsch to try to keep my next-youngest sib back, tried to get a repeat of 3rd-gr because she could see he was not reading– but already (in ’63) there was that ideology that kids should not be held back… He ended up having to to repeat 9th-gr (!), which would have been anathema for most kids, but my bro had preternatural social skills… This kid ended up not being able to graduate hisch despite attempts at comm-coll & GRE,..
BUT I have to share that this next-youngest bro was so talented in trades that at 14 he wired my playhouse for electricity & cut glazed windows & installed running water!
When Mom saw that my bro wouldn’t grad hisch & thus not do college, she arranged w/our former asst-parish-priest (an Indian who had returned to his Bombay college as microbiology prof– this was during an Indian economic bust, when maint staff were unaffordable–) for my 18-yr-old bro to design & teach a one-year course for incoming frosh to maintain their own lab instruments, which skills they would pass on to the next incoming class etc
This was my Mom’s idea for a viable substitute for a college-ed for my bro, helping him understand his particular gifts & broadenening his world-view. He went on to success as a self-employed tradesman.
When my dyslexic sis came through K12 7 yrs later (late ’70’s), her LD was diagnosed early, & she was afforded much treatment in mid/hisch thro an IEP. She was inspired to take a BS in SpEd teaching, got her Master’s, & was named teacher of the year for Bkln/Staten Island her first yr. She has continued to excel in SpEd, & as union rep, & in admin [esp excels in wkg w/delinquents] & now holds an asst principal position in a prestigious upstate-NYS hisch.
Thank you for sharing. Your family history is proof that challenged learning labels are not a sentence to fail.
Both of my parents were high school dropouts but both were avid readers. I was held back in 1st grade for an extra year. When the so-called district’s experts tested me and judged me to be so retarded I’d never learn to read or write, my mother broke into tears and then went to my 1st-grade teacher for advice – the teacher did not agree with the test experts.
Following that teacher’s advice, my mother taught me to read at home. Twelve years earlier, my mother heard the same thing about my older brother who never learned to read, ended up spending 15 years of his life in prison and struggled to survive on mostly poverty wages until his death at 64. He drank heavily, did drugs, smoked, and worked hard labor jobs for under the table pay that was usually below minimum wage. Being paid under the table meant he didn’t have to declare the money and pay tax on it. To make ends meet, also tax-free, he spent hours in his old truck driving around collecting cardboard, glass, and aluminum cans to sell to recycling centers.
A few years before my brother died, a broken worn out man old before his time, he told me he’d rather be dead than be sent back to the slammer. That’s what he called prison.
The slammer.
You really wonder how much being told he was stupid affected his life. Actually, I don’t wonder at all. It is a rare individual who can beat the odds in the face of such dismissal.
The tragic world of high stakes testing makes it a horrible time to be a child in America. This NAPLAN sounds like napalm, equally destructive.
Thanks, Diane!
Phil and retired New Zealand educator, Allan Alach who has taken over the Treehorn Express function and I have been in communication over the last few weeks. We are going to serialize my book “Infidelity to Truth: Education Malpractice in American Public Education” on the Treehorn site/email list. Also, I should have the hard copies of the book next week, but probably will not start selling them until I get everything set up which won’t be until after the 5th or so of July. Will let all here know when they are available.
It’ll take a bit to get set up but I won’t be available anyway at the end of this month until early July because I will be helping a life-long friend get started on his journey down the Missouri River from Yankton, SD to St. Louis, MO. It’s about a month’s worth of canoeing 40-50 miles a day. I’ll be driving his truck back to eastern MO and catch up with him towards the end when he comes by my place which is only a few miles north of the Missouri River.
Phil is a brilliant, humorous commentator, even above the likes of Some Damn Poet and Krazy TA-and they’re the bestest! Sorry, folks but the Aussie lingo and humor just puts him a notch above.
The Treehorn Express can be found at: https://treehornexpress.wordpress.com/
Tell your friend to say hello to Indian Cave state Park in Nebraska. Also wave at the Niobrara River as he goes by. Two of my favorite places. My wife and I have a few more favorite spots in Missouri as well.
Will do!
Roy,
He’s putting in at Yankton, so he’ll miss the Niobrara, but he will definitely go by Indian Cave. I imagine it will be one of the rendezvous points as another one of our lifelong friends, who is an excellent photographer that recently completed a cancer treatment regimen and is looking forward to the trip, will be traveling with him via road.
The gentleman (and I use that term lightly) doing the floating is using his own hand-made wood strip/fibreglass canoe-he’s made two and a kayak, beautiful rivercraft! But then the three of us were part of a scout troop that made two 16′ wooden strip fibreglass canoes in scouts when we were in 7th & 8th grade. I still have one of them, now 50 years old, in the barn but it needs work before being seaworthy again. Oh, the stories we can tell about the adventures in that canoe!! I used to transport it on the top of my VW bug!
I hope you all have fun.
Way to go, Senor! 🙂
Hmmm. . . .
Generally speaking the only ones who call me Señor are my former students. LG, are you a former student of mine?
Nawww…just a fellow poster who sometimes pays attention. Haha!
¡Qué bueno ensayo! ¡Les deseo mucha suerte!
Me gusta.
When I was in 7th grade, the district started a new program in foreign language. We all took some test that was supposed to identify those who were best able to take Spanish. My parents, who were friends with the superintendent (my father was BOE president), told him that I was not to be placed in the class unless I tested into it. I didn’t, but the supe put me in anyway without telling them I had not tested in. At the end of the course, I was second behind the supe’s daughter who had lived in a Spanish speaking country. That is when he told them the results.
Like!
Good for you, speduktr, and the Superintendent, who apparently knew more about you, and the program, than your test score would indicate. And apparently had more confidence in your ability to succeed in the Spanish program than your own parents did!
I guess your parents didn’t want to be seen as “pushy” parents who got their kids into such programs because they “knew somebody” (or some such- at least, I assume this is why your folks said what they said).
“I guess your parents didn’t want to be seen as “pushy” parents who got their kids into such programs because they “knew somebody” (or some such- at least, I assume this is why your folks said what they said).”
Yup. It just wasn’t in their DNA. I was very lucky to have them as my parents although, of course, I didn’t know it while I was growing up.
We seldom appreciate our parents while we are growing up, and then we learn to value them and the lessons they taught us later.
As Mark Twain said,
“When I was a boy of fourteen, my father was so ignorant I could hardly stand to have the old man around. But when I got to be twenty-one, I was astonished at how much the old man had learned in seven years.”
My mother got really smart after I had my first child. 🙂
They also bought into testing as a measure of ability like probably close to everybody at that time. Interestingly enough, by the time my youngest sister was applying to college, I remember my mother talking about how she was a terrible test taker who routinely out performed expectations. She was the best student of all of us not in terms of grades (although I couldn’t tell you what they were) so much as in her approach to learning. With me, you could always tell when I was really interested in something because I put more effort into those classes. My parent recognized those things about us, and would have acknowledged Duane’s championing of Wilson’s thinking as having merit.
How creepy that there was ‘some test’ to ferret out for-lang ‘aptitude’. Why: limited seats available??
I was very fortunate to be in a small, rural collegetown primary school during the brief, Sputnick-inspired assay into early-foreign-language-learning. We all got lunchtime French-lessons for a couple of years. That inspired me, & got me a leg up when 7th-gr French came along.
“Why: limited seats available??”
It was a pilot program. You know, those small scale experiments that wouldn’t break the bank if they failed?
like
I love Phil’s sign off:
“I’m still caring for the kids. Are you?”
By the way Phil Cullen will turn 90 this year. His education work is second to none and the accolades he as earned/garnered along the way are amazing including the Medal of Australia award given by the Queen.
Duane, have always loved your Wilson “rant,” & often commented that it couldn’t be repeated enough (or ever be redundant, bearing repeating), especially with new Diane Blog readers. Glad to see you’ve made it “down under.” I am very much looking forward to seeing you on your book tour in the near future.
Of course, it’ll have to wait till after your great adventure.
Happy trails, & thanks for all your most valuable comments and incredible wisdom.
For sure, you’re a son of the “Show Me” state!
Thanks for the very kind words, rbmtk!! También te amo.
My part in my friend’s great adventure (can’t but help to think of PeeWee with that term), is easy, to ride up to Yankton SD, hang out for a few days then drive his truck back here. Pretty easy and relaxing.
My “great adventure” will be in the fall driving out to Oakland for the NPE conference. I’m taking 4-5 days to drive to Seattle, camping along the way (I’m a rather cheap old bugger) to hang out with my daughter (the bestest there is) for the weekend, then take 4-5 days driving and camping through Oregon and northern California, then down to Oakland for the conference. Then hit Tahoe where a buddy lives, down to Las Vegas where another couple live, and then down to Phoenix where another friend lives, after that the drive back to Missouri.
Hey what else is there to do when one is retired? Maybe I can do that “book tour” along the way. Hmmmm….!
Oh–meant to say–so glad to hear all that about Phil–an advance happy birthday, Phi–you’re an inspiration to us all.l
The first installment of the book is a brief introduction of myself and what the book is about: https://treehornexpress.wordpress.com/2017/06/23/infidelity-to-truth-education-malpractices-in-american-public-education/