Gerri K. Songer, a literacy specialist in Illinois, here explains what is wrong with the Common Core tests:
I was asked by my EA President and the Superintendent of IL HS Township Dist. 214 to review Smarter Balanced, ACT, SAT, and PARCC. The following is a portion of my review:
“In terms of text complexity, ACT, SAT, and PARCC all use excessively high level text. PARCC is by far the worst assessment for many reasons, some of them including the use of multiple passages between which comparisons and contrasts are made; finite detail-oriented questions; and multi-step cognitive analysis. Yet, the ACT disseminated last March resembled PARCC in reading and mathematics, with the exception of multiple passage comparison/contrast. If the agenda of both ACT and SAT is to become more like PARCC, then one, in essence, wouldn’t be any better than another.
I’m still going through the SAT materials, so I’m not able to make any conclusions about this assessment yet. I don’t see anything strikingly different in Smarter Balanced, other than the listening portion of this assessment. Like PARCC, it contains multi-passage comparison/contrast, but at least the text used in these comparisons is shorter. Text is still excessively high. One significant difference ACT has over other assessments is the use of the following scaffolding: http://www.act.org/standard/planact/english/index.html This format is easier for teachers to work with, and it helps them target individual skills on which to focus in different level courses and grade levels.
There is no research I have come across that supports the use of archaic vocabulary used in primary source documents such as the Declaration of Independence to “level the playing field” in terms of comprehension. In fact, research supports the opposite. The single most important component of reading comprehension is background knowledge. Even when students cannot understand vocabulary terms used in a reading passage, they can still glean meaning from text using context to compensate for words they don’t understand.
Using archaic vocabulary only favors high achieving, high socio-economic students who have the fortitude and patience to weed through confusing, complex, and unfamiliar text. To understand this from the students’ point of view, I have to ask myself, how intelligent would I appear if I were assessed using text written in Spanish? I know some Spanish, but I’m not fluent in it, and such an assessment certainly wouldn’t appropriately or adequately assess my ability to compare, contrast, synthesize, apply, etc., information for purpose of extracting meaning.
Not only do these assessments not assess what they claim to assess, but I’m also convinced, based on brain research, they are actually harmful to students. The brain only has so much neural support. If the brain is trained through repetition to narrow this neural support to a specific region of the brain, then neural activity will supply less support, or perhaps no longer support, other very important areas of the brain, specifically those areas allowing for the ability to think conceptually and creatively.
Ray Charles was born with sight, but lost his sight early on in his childhood. Once he lost his sight, his senses of hearing and touch became more acute. This happened because neural activity once supporting sight was redirected to support other senses – hearing and touch. Without sight, there was no need for neural activity in this region of the brain, so neurons travelled to other areas that did need support. Fortunately, genius for Ray Charles evolved through his auditory modality in the form of musical, artistic expression.
It is exceedingly concerning that our assessment practices could likely be obstructing the natural development of human thought processes, and my heartfelt message is that this isn’t a question of what test is better or worse – this is an issue of morality and calls for careful consideration as to what we as educators are doing to our students in our effort to neatly package their performance into statistical boxes that are misleading, at best, and that lie, at worst. We are using quantitative assessment to evaluate qualitative data – it simply cannot be done. We, as mature adults, are far more advanced than what our cognitive abilities indicated as adolescents.
Unfortunately, government is dictating educational practice, but perhaps it’s time to evaluate the government’s ability to determine what sound educational practice is. The original intent behind the use of standardized assessment was a noble one, but it has spun out of control, and current research suggests it may actually be detrimental to student learning and damaging to the neurology of the brain.
My best advice is to “take the path less traveled by;” Robert Frost claims it “made all the difference.”
I’ve always believed students were the educators top priority, even if this means making very difficult decisions with which many may disagree. Funding is not a priority if it comes at the expense of our students’ well-being. They are in our care, and we, as adults and as educators, are supposed to know and do what is “educationally” sound for them.
We make mistakes, we learn from them, and then we adjust accordingly. We aren’t perfect, but when there is strong evidence indicating our assessment practices are very likely damaging to the natural development of neural activity in the human brain, we should stop what we are doing until this evidence is analyzed through appropriate research. My bet is this could be as simple as speaking with doctors specializing in the neurology of the brain.”

I completely agree with this author. Asking students to think, critically, is completely unfair. I mean why should we expect them to actually learn complex skills and be able to apply them? That would lead to them holding valuable jobs and paying taxes. What we (read unions and progressives) really want are a bunch of mindless automatons who will leech off the government and supply votes for our progressive agenda. That is the purpose of schools these days. To inculcate unwitting students into the religion of “social justice”.
LikeLike
As always, you have reading comprehension issues. Try a close reading – Coleman would be proud of you.
LikeLike
Riiiight, who wants progress?
LikeLike
It appears, virginiasgp, you have been brainwashed by FOX News.
LikeLike
Wrong thread?
Wrong blog?
Wrong reality?
In the spirit of CCSS closet-reading of decontextualized informational text, maybe y’all have not taken this in the spirit in which it is offered.
😏
With the aid of a few more flashlight batteries—and more than a few hours reflection—I see a call to follow doctor’s orders.
¿😳?
“A day without laughter is a day wasted.” [“Dr.” Charlie Chaplin, expert in Laughology]
This day has not been wasted.
😎
LikeLike
As always, thanks for the great quotes and a smile.
LikeLike
TC: thanks for the thanks, but I must respectfully ask you to proffer them to the person who did all the heavy lifting…
I am powerless to poke fun at the rheephormistas. They do a better job of it then almost anyone I read or hear, one of the few exceptions being SomeDAM Poet, the wordsmith of this blog.
To mix the serious with the ridiculous: I sometimes refer to the enforcers and apologists of the self-styled “education reform” movement as the “sneer, jeer and smear crowd.” Yet add in an attitude that, to paraphrase Mark Twain:
“All you need in this life is ignorance and confidence, and then $tudent $ucce$$ is sure.”
How powerful a toxic mix is that? Sometimes an old dead French guy can top even very old and very dead and very Greek guys:
“Ridicule dishonors a man more than dishonor does.” [François de la Rochefoucauld]
What would he have said about self-ridicule?
😎
LikeLike
Let’s see the evidence on applying high standards to disadvantaged students. Yes, it does appear that the notion that disadvantaged kids (1/3 of whom are 4 years behind when they entered Thurgood Marshall Academy) can achieve progress on rigorous standards is completely unfounded. I mean this charter school in the article must be making up the data. it can’t possibly be that such kids can actually learn and compete. Why, that would undermine the whole anti-union, opt-out activist agenda of the unions.
So let’s see what you are really saying. Surely you don’t suggest that affluent kids who score well on PARCC can’t handle the Common Core curriculum. I want my kids to be challenged (currently they are not in Virginia) and rigorous CC does the trick. So you seem to be suggesting that disadvantaged kids are the one who can suffer irreparable harm if they are introduced to challenging standards. You seem to be suggesting that holding disadvantaged kids to the same standards as the affluent will cause psychological and even physical damage to their brains. That sounds an awful lot like blatant racism! But rather than describe your position for you, why don’t I let you answer the question.
Do you think providing the same challenging standards to disadvantaged minority kids will cause them damage when affluent students are able to excel using those same standards? Are you all simply closet racists?
LikeLike
Virginiasgp, look at the results of any standardized test–whether the SAT or ACT or NAEP or state tests. The scores are arrayed by income. Have you ever wondered why? On the SAT, the kids whose family income is over $200,000 are at the top, the kids whose family income is under $20,000 are at the bottom. Please explain.
LikeLike
Diane, while income does play a role in these results, studies have shown that much of that correlation is due to aptitude (IQ). Indeed, studies have shown that if you separate identical twins at birth, more of the variation in test scores later in life is due to their native aptitude than to the income of the household in which they were raised.
Thus, nobody is suggesting that we evaluate teachers on whether they teach the kids of Neil deGrasse Tyson (an astrophysicist) or those of a construction worker. That is why SES is taken into account in VAMs. VAMs based on Common Core tests measure the contribution of the teachers/school/etc to the growth of similarly situated kids.
The scores are also arrayed by parental education. But not always. I can show you results in which the children of high school grads exceeded the results of children of graduate degree recipients. It all depends on how much income/parental education are tied to the students’ aptitude.
Diane, for one who touts “correlation doesn’t mean causation”, it’s a little disingenuous to try to bring up the income correlation, don’t you think? I guess you are running short on excuses.
LikeLike
Virginiasgp,
If VAM is so great, why has it worked NOWHERE? Billions of dollars have been spent for VAM, and every state to date has reported 95-97% of all teachers are rated effective or highly effective. What a monumental waste of money that should have been used in the classroom.
LikeLike
Diane, the solution is within your grasp! Previously, states reported 99% effective. If they now retrain and/or remove that 5% they can now identify, then we could achieve the gains that Hanushek predicted. You see, justice for those disadvantaged students sitting in the ineffective teachers’ classrooms is now with reach. Not the imaginary “social justice” of your supporters, but a real opportunity to obtain an equal education. Me thinks the court said as much nearly 60 years ago in Brown v Board of Education.
LikeLike
Virginiasgp, that’s silly. We lose large numbers of teachers every year. The VAM is junk
LikeLike
Virginiasgp,
You may have missed this comment:
“The single most important component of reading comprehension is background knowledge. Even when students cannot understand vocabulary terms used in a reading passage, they can still glean meaning from text using context to compensate for words they don’t understand.”
Exactly. This is why middle class children struggle less with reading. Their families, who are fortunate enough to have the time and the money, take them away on vacation, take them to museums, read to them on a regular basis, sign them up for camp in the summer, and enroll them in extra art, music, dance, drama, science classes after school, etc. Not to mention that families themselves are educated and bring their knowledge of the world home. This is what builds background knowledge! No amount of “close reading” is going to compensate for what children living in poverty are lacking. And this is why it is essential that those children are exposed to art, music, dance, drama, science while in school. To think that David Coleman and all the other reformers don’t understand this, is ludicrous.
LikeLike
Diane, I understand that disadvantaged children have less variety in their experiences and their parents talk to them significantly less on average. But the solution to that is … less rigorous standards? I think not.
I have described the situation of a rural Georgia classmate of mine in college. He had never seen logarithms prior to college and was enrolled in Calculus I. It was a rude awakening for him and his initial grades were less than what they could be. But he caught up and fast. The solutions to less exposure is more exposure through the curriculum. If a poor student doesn’t know what a “yacht” is (often used as an example of upper vs lower SES knowledge), being exposed to that word in school increases his overall background knowledge. In other words, students’ exposure increases when using rigorous standards. That’s exactly why each teacher hung his/her alma mater’s emblem in the class – so that students would be exposed to colleges that had never dreamed of.
Middle class students will gain a wide variety of experiences outside school. Disadvantaged kids will mostly gain these experiences through reading (many wonderful journeys around the world and beyond) and via interaction in class with teachers and fellow students. Exactly, how else were you envisioning these students “catching up”? Oh, that’s right. Some folks on here want to pay for all the students to take international trips and have public funds used to provide every child an “equal experience”. I’m guessing that won’t cost much, maybe $2T! Or maybe we can hold students to high standards.
Btw, tell me this. How many of our teachers have a “broad variety of experiences”? How many have traveled abroad? How many have worked in other fields? Not nearly as many as you would think. If we would encourage career switchers without onerous credential reqts and tell the public what teachers actually earn (including 20% pension contribution), we would get more teachers with diverse experiences. I gather you are all for lowering credential reqts and publishing private sector equivalent pay then?
LikeLike
Virginiagsp you are absolutely right that ‘disadvantaged kids will mostly gain these experiences through reading’– & I’ll pile on middle-class kids as well (like me, whose 2 working parents had precious little time for family trips). However my reading of CCSS-ELA 3rd-8th suggests this important reading will be narrowed, not broadened as necessary to reach every kid whose IRL experiences are narrowed by economic circumstances.
My middle son’s [mid-’90’s] 1st-grade class turned him from a non-reader to a voracious reader. The first 1/2 of the year was focused on teacher-directed reading skills. In the 2nd semester, the classroom was a virtual library of texts grade-levelled in bins from beginner to 3rd-gr and up– drop everything and read was a daily feature.
The teacher was nearing retirement, often satirized as a ‘bad teacher’ because she nodded off while her kids were reading. Some only made it to to 2nd-gr level, which was perfectly acceptable. My kid was reading ‘goosebumps’ & ‘choose-your-own-adventure in 2nd, & Harry Potter’ over the next few grades, thanks to her. He is not by nature a bookworm, yet had the reading skills to keep up with advanced lit selections thro h.s., & to this day stays on top of his field thro wide reading on the internet.
That teacher could not conduct her 1st gr class in that manner today thanks to CCSS-ELA & its assessments. Fed curriculum/ testing policies, bought into by our NJ gov, impose their supposedly-superior knowledge of curriculum on my neighborhood school. That means my neighbors’ first-graders will or will not become enthusiastic readers by virtue of one specific way of teaching 1st grade.
LikeLike
Nary a day should go by without me asking you for the answers to questions that you refuse to answer, so. . . here’s today’s, only the fifth time I have requested that you answer these very fundamental educational questions (without which your love affair with VAM/SGP is actually puppy love*):
Please explain the standard(s) involved in these supposed measures. Where is that standard? How did it come about? Is there only one standard? Who determined that standard? Does the process involved in making and using these supposed standards and measurements follow OSI protocols for making and using standards? If so where may I find that justification?
LikeLike
Duane Swacker yet again ventures off on an irrelevant diatribe. There are NO standards in education that follow OSI protocols. None created by teachers. None created by administrators. None created by unions. Thus, your whole point is irrelevant. We don’t need to measure to 6 sigma to understand which teachers are effective.
Note that all of the current evaluation methods for teachers are more variable than any VAM method. The MET study showed that variation between evaluators were greater than even the variation between lessons using the same evaluator. Surely, teachers must teach certain subjects better than others. But evaluators simply can’t agree on which teacher is effective by observing them. Thus, the premise that the current evaluation system works is ludicrous. The question is why wouldn’t we replace it with a more consistent VAM model that has been shown to have lifetime income effects. The observation model never has been shown to have those effects.
But let’s pull the string on your ridiculous theory. Are teachers’ own tests/exams that they administer to students based on “standards”? Based on irrefutable standards using the OSI protocols? Of course not. Thus, teacher evaluations of students are completely unreliable and should not be allowed to be issued. In fact, the grades given out by teachers have lifetime effects on their students since gpas are used by colleges, scholarships and employers. Duane Swacker, if you can’t demonstrate that all teacher-given exams are based on proven standards, then you MUST call for their discontinued use! Are you really suggesting that the “unreliable” tests given by teachers be banned?
In the end, it all comes down to whose side are you on. We can never be 100% certain that a given teacher is completely ineffective or effective. That’s life. We get it. You all want to take the side of the teacher every single time at the expense of students. As I’ve said before, if there were an 85% chance that a teacher were ineffective, you all want disadvantaged kids to be forced to remain in his/her classroom than to take the side of the students and to replace that teacher. Your position in unconscionable.
LikeLike
Yes, I argue that teacher grades are also invalid. As a matter of fact I discuss “grading/grades” with my students at the beginning of the year and let them know that they-grades are bullshit, one of the big lies told often enough that almost all believe in. And no, I don’t use the term bullshit, I use excrement of bovine origin since I teach in a rural community all understand. GRADES are a BIG LIE and I do not refer to/emphasize them in discussing with students where they were in their learning only in regards as to what the students need to do get a certain grade since we are forced to play that game.
And teacher tests suffer all the problems standardized testing suffer from. Do I use student-taken written assessments in helping the student to understand what they may know or may not know? Of course. But the results are not used as a statement about the students but as a means for going back over the material and hopefully helping the students have a more accurate picture of their learning Spanish. So no, I am not calling for an outright ban, not into that kind of domineering outlook on things, on teacher made assessments/tests. Just calling for a more refined usage of the information a student gleans from said test and not labeling a student from the results.
Thanks for correctly answering my questions/concerns. However they are not part of an “irrelevant diatribe”, the questions and answers get to the heart/foundation/crux of the fundamental invalidity of the VAM/SGP process as those VAM/SGP results are primarily based upon standardized test scores. Your idiology blinds you to the fact that if the foundational concepts of anything are non-existent, as you have just pointed out that there are no standards and no measuring devices upon which those tests are based, then any conclusions drawn from that non existent base can only be chimeras, duendes, figments of your imagination even though those figments are given life due only to your imagination.
To continue to believe in those chimeras in the face of evidence that completely rationo-logically destroys them is the height of idiocy, thus your idiology has to take over to cover your lack of credibility in pushing the illogical, invalid, unethical and immoral VAM/SGP delusions.
“Your position in unconscionable.”
Your putting words into my keyboard by insinuating that I agree with what you stated about ineffective teachers is what is unconscionable. I have never stated anywhere near such a thing.
Basta ya contigo. (Unless you keep spouting the same crap about VAM/SGP and so-called “ineffective” teachers.)
LikeLike
I readily admit I am not this inventive—
“the whole anti-union, opt-out activist agenda of the unions.”
Word salad. Cognitive dissonance. Topped off with a hearty helping of obliviousness.
TC: I rest my case.
😎
LikeLike
The author is talking about the quality of the assessments. If you want to see mindless automatons continue with the scripted garbage being pushed as preparation for creative and critical thinking. We don’t even have to address whether critical thinking is being tested.
LikeLike
2old2teach (thank goodness btw), if you don’t think critical thinking is being measured by these multiple choice tests, answer this question:
In Let’s Make a Deal, the show would provide a contestant with a choice between 3 doors with prizes behind each. However, there were certain standard rules that followed:
1. Only one door had a valuable prize. Without exception, the other 2 doors would have a silly prize such as a goat or pig.
2. Once you selected your choice of door initially, the host would reveal what’s behind one of the other doors. However, he would always reveal a door with the silly prize like a goat. He would never reveal the valuable prize like a vacation.
3. Once you had chosen your initial door in (1) and the silly prize had been revealed in (2), you were given the chance to switch doors. For example, if you picked Door #1 and the host revealed a goat behind Door #2, you could now stay with Door #1 or switch to Door #3.
What should you do? Stay with Door #1 or switch to Door #3? And what are the consequences of each:
A. Staying with Door #1 gives you a better chance of winning (2/3 chance). Stay with Door #1.
B. There is no difference in chances and are equally likely to win either way (50% chance)
C. Switching to Door #3 increases your chances of winning to approx 60%. Switch to Door #3.
D. Switching to Door #3 increases your chances of winning (2/3 chance). Switch to Door #3
See, this measures that vaunted probability that folks were talking about earlier and obviously critical thinking skills. But maybe you are 2 old 2 answer it eh? Let’s see.
LikeLike
The key sentences for me:
“It is exceedingly concerning that our assessment practices could likely be obstructing the natural development of human thought processes, and my heartfelt message is that this isn’t a question of what test is better or worse – this is an issue of morality and calls for careful consideration as to what we as educators are doing to our students in our effort to neatly package their performance into statistical boxes that are misleading, at best, and that lie, at worst. We are using quantitative assessment to evaluate qualitative data – it simply cannot be done.”
The tail of standardized testing is wagging the dog of genuine learning and teaching. Bad for students. Bad for teachers. Bad for almost everyone one else.
But those that mandate worst pedagogical practices for OTHER PEOPLE’S CHILDREN do the opposite for THEIR OWN CHILDREN.
So-called “education reform”: double talk; double think; double standards.
Just my dos centavitos worth…
😎
LikeLike
Couldn’t agree more with the morality and the qualitative v quantitative points.
Class warfare is the only explanation that makes sense.
LikeLike
“Asking the Wrong Question”
“What is right with CC test?”
That’s the question that is best
“Nothing” for the ELA
“Empty set” for math, oy vey
LikeLike
“A Plinth of Space”
A plinth of space
Is Common Core
An empty base
Of Coleman lore
“Bowdlerdash”
The Pearson tests are bowdlerdash
Ephemeral nimbus plinths
They’re really an insufferable hash
Enough to make you wince
LikeLike
Remember the “ephemeral indoor nimbus clouds” inside a defunct prison/psychiatric hospital in Belgium?
LikeLike
“Pearsonal Integrity”
“Integrity of Pearson Test”
Is oxymoron at its best
To test a standard that is junk
Is nothing more than purest bunk
LikeLike
Again … LOVE your poem. Right on.
LikeLike
Recharge your batteries, perhaps I can help.
Stopping by a test site on a snowy evening (Sorry Mr. Frost)
Whose tests these are I think I know
But in the trash they still shall go
My opt out form is right by me here
To see Coleman, his rage will flow
The ACT and SAT, all so very weird
The SBAC and PARCC like crap, smell very near
Pearson says pofits are down, more testing they must steer
Coleman rants foul and profane, the darkest of the year!
Our shackles we now give a shake
All know these tests are a big mi$take
We hear the children as they weep
As teachers feet ot the doors sweep
The woods are lovely, there we’ll teach
Our school the tests can no more reach
Just like Sidwell’s and the Lab
They don’t yield to reformist fads
Skills and talents to nurture and find
Our schools are more than a testing grind
Retired or not we’ll persevere
All childrens spirits and minds, not just some
we revere.
LikeLike
I have always found open ended questions on content to benefit poor minority students. I managed to get many ELLs through the old NYS elementary science, social studies and math tests. Even though their grammar and vocabulary were poor, they managed to explain their thinking and got at least partial credit for answers. Bubble tests are a mine field of obfuscation and complex misleading wording and sentence structure that are confusing to ELLs. They are less able to show what they understand in the standardized test format as a result their scores are lower.
LikeLike
Your post is true on so many levels. How else to ascertain what the student is thinking? Without ascertaining that, how else to teach him how to express his thought so that it reaches others?
I had.an IEP son who was obviously gifted in speaking, eventually writing– but also had an advanced grasp of mathematical concepts. This was a kid who at 13 could explain to this math-brainless mom what quadratic equations were, what they were good for & how to work them– but LD’s prevented him from working one in writing to its conclusion. It took an unusually gifted 8th-gr math teacher (after yrs of failing math grades) to perceive that ability. She worked with him daily for months to help him cross the bridge into writing it down & passing the test.
LikeLike
“The single most important component of reading comprehension is background knowledge. Even when students cannot understand vocabulary terms used in a reading passage, they can still glean meaning from text using context to compensate for words they don’t understand.”
Exactly. This is why middle class children struggle less with reading. Their families, who are fortunate enough to have the time and the money, take them away on vacation, take them to museums, read to them on a regular basis, sign them up for camp in the summer, and enroll them in extra art, music, dance, drama, science classes after school, etc. Not to mention that families themselves are educated and bring their knowledge of the world home. This is what builds background knowledge! No amount of “close reading” is going to compensate for what children living in poverty are lacking. And this is why it is essential that those children are exposed to art, music, dance, drama, science while in school. To think that David Coleman and all the other reformers don’t understand this, is ludicrous.
LikeLike
I agree with the author of the article, but I would like to see the cites if, for no other reason, to read the cites myself.
Dr. Bickenheuser, Ed.D., O.F.S.
LikeLike
I agree, the article would be tremendously benefited by actual reading passages, test questions, etc to support his conclusions.
LikeLike
It is premature to suggest that we have anything else but a rudimentary understanding of learning from what we observe and what actionable information is actually available from brain research and fMRI. We know a lot more from simply observing behavior. For some reason, such information does not carry the same weight as the electrical activity in the brain that we can watch on a screen. I don’t mean to denigrate such research but to suggest that it should be viewed as the definitive avenue for understanding behavior or what to do about it at this point in time borders on pseudoscience.
LikeLike
From Songer: “this is an issue of morality and calls for careful consideration as to what we as educators are doing to our students in our effort to neatly package their performance into statistical boxes that are misleading, at best, and that lie, at worst. We are using quantitative assessment to evaluate qualitative data – it simply cannot be done.”
Not only is the false labeling and sorting and separating students a moral issue but also an ethical one. That’s two strikes against it there. The third, and it’s a 105 mile inside high and hard one that the edudeformers don’t even come close to catching up to with their feeble and futile swing is that those assessments, standardized tests and the foundation upon which they are built (and by definition any usage of those results like VAM/SGP) are COMPLETELY INVALID. To better understand why read what critics label as “existential nonsense” and “post-modern claptrap” (to no avail because their feeble brains can’t even comprehend what Wilson is saying) the never refuted nor rebutted most important work of post-WW2 American (even though it was written in Australia) education, Noel Wilson’s “Educational Standards and the Problem of Error” found at: http://epaa.asu.edu/ojs/article/view/577/700
Brief outline of Wilson’s “Educational Standards and the Problem of Error” and some comments of mine.
1. A description of a quality can only be partially quantified. Quantity is almost always a very small aspect of quality. It is illogical to judge/assess a whole category only by a part of the whole. The assessment is, by definition, lacking in the sense that “assessments are always of multidimensional qualities. To quantify them as unidimensional quantities (numbers or grades) is to perpetuate a fundamental logical error” (per Wilson). The teaching and learning process falls in the logical realm of aesthetics/qualities of human interactions. In attempting to quantify educational standards and standardized testing the descriptive information about said interactions is inadequate, insufficient and inferior to the point of invalidity and unacceptability.
2. A major epistemological mistake is that we attach, with great importance, the “score” of the student, not only onto the student but also, by extension, the teacher, school and district. Any description of a testing event is only a description of an interaction, that of the student and the testing device at a given time and place. The only correct logical thing that we can attempt to do is to describe that interaction (how accurately or not is a whole other story). That description cannot, by logical thought, be “assigned/attached” to the student as it cannot be a description of the student but the interaction. And this error is probably one of the most egregious “errors” that occur with standardized testing (and even the “grading” of students by a teacher).
3. Wilson identifies four “frames of reference” each with distinct assumptions (epistemological basis) about the assessment process from which the “assessor” views the interactions of the teaching and learning process: the Judge (think college professor who “knows” the students capabilities and grades them accordingly), the General Frame-think standardized testing that claims to have a “scientific” basis, the Specific Frame-think of learning by objective like computer based learning, getting a correct answer before moving on to the next screen, and the Responsive Frame-think of an apprenticeship in a trade or a medical residency program where the learner interacts with the “teacher” with constant feedback. Each category has its own sources of error and more error in the process is caused when the assessor confuses and conflates the categories.
4. Wilson elucidates the notion of “error”: “Error is predicated on a notion of perfection; to allocate error is to imply what is without error; to know error it is necessary to determine what is true. And what is true is determined by what we define as true, theoretically by the assumptions of our epistemology, practically by the events and non-events, the discourses and silences, the world of surfaces and their interactions and interpretations; in short, the practices that permeate the field. . . Error is the uncertainty dimension of the statement; error is the band within which chaos reigns, in which anything can happen. Error comprises all of those eventful circumstances which make the assessment statement less than perfectly precise, the measure less than perfectly accurate, the rank order less than perfectly stable, the standard and its measurement less than absolute, and the communication of its truth less than impeccable.”
In other word all the logical errors involved in the process render any conclusions invalid.
5. The test makers/psychometricians, through all sorts of mathematical machinations attempt to “prove” that these tests (based on standards) are valid-errorless or supposedly at least with minimal error [they aren’t]. Wilson turns the concept of validity on its head and focuses on just how invalid the machinations and the test and results are. He is an advocate for the test taker not the test maker. In doing so he identifies thirteen sources of “error”, any one of which renders the test making/giving/disseminating of results invalid. And a basic logical premise is that once something is shown to be invalid it is just that, invalid, and no amount of “fudging” by the psychometricians/test makers can alleviate that invalidity.
6. Having shown the invalidity, and therefore the unreliability, of the whole process Wilson concludes, rightly so, that any result/information gleaned from the process is “vain and illusory”. In other words start with an invalidity, end with an invalidity (except by sheer chance every once in a while, like a blind and anosmic squirrel who finds the occasional acorn, a result may be “true”) or to put in more mundane terms crap in-crap out.
7. And so what does this all mean? I’ll let Wilson have the second to last word: “So what does a test measure in our world? It measures what the person with the power to pay for the test says it measures. And the person who sets the test will name the test what the person who pays for the test wants the test to be named.”
In other words it attempts to measure “’something’ and we can specify some of the ‘errors’ in that ‘something’ but still don’t know [precisely] what the ‘something’ is.” The whole process harms many students as the social rewards for some are not available to others who “don’t make the grade (sic)” Should American public education have the function of sorting and separating students so that some may receive greater benefits than others, especially considering that the sorting and separating devices, educational standards and standardized testing, are so flawed not only in concept but in execution?
My answer is NO!!!!!
One final note with Wilson channeling Foucault and his concept of subjectivization:
“So the mark [grade/test score] becomes part of the story about yourself and with sufficient repetitions becomes true: true because those who know, those in authority, say it is true; true because the society in which you live legitimates this authority; true because your cultural habitus makes it difficult for you to perceive, conceive and integrate those aspects of your experience that contradict the story; true because in acting out your story, which now includes the mark and its meaning, the social truth that created it is confirmed; true because if your mark is high you are consistently rewarded, so that your voice becomes a voice of authority in the power-knowledge discourses that reproduce the structure that helped to produce you; true because if your mark is low your voice becomes muted and confirms your lower position in the social hierarchy; true finally because that success or failure confirms that mark that implicitly predicted the now self-evident consequences. And so the circle is complete.”
In other words students “internalize” what those “marks” (grades/test scores) mean, and since the vast majority of the students have not developed the mental skills to counteract what the “authorities” say, they accept as “natural and normal” that “story/description” of them. Although paradoxical in a sense, the “I’m an “A” student” is almost as harmful as “I’m an ‘F’ student” in hindering students becoming independent, critical and free thinkers. And having independent, critical and free thinkers is a threat to the current socio-economic structure of society.
LikeLike
Virginiasgp, this is partly in response to your initial reaction, but also contains my reactions and questions.
I grant you that it’s difficult to reply cogently to an excerpt which is heavy on generalization and absent specific reading passages/ test questions. If I were the recipient, I’d be requesting lots of back-up for the opinions stated.
But you reply only to this line “In terms of text complexity, ACT, SAT, and PARCC all use excessively high level text.” The writer does not define ‘excessively high level’ in the context of the standards addressed by the assessment. Although he does say later ‘these assessments not assess what they claim to assess.’ My inference is that the assessment’s level of text is of a higher complexity than that suggested by the standards for the level tested. I don’t think we can conclude that the writer simply wishes to lower the bar for poor kids.
The writer makes other points not covered in your blanket dismissal.
I agree with this one: he questions ‘the use of multiple passages between which comparisons and contrasts are made.’ ‘Compare and contrast two reading passages’ is good material for a 1-page paper requiring an hour or so of concentration. Such an exercise has no place in a timed test of two hours or less. I agree particularly in the context of online computer tests, having trialled PARCC ELA questions. The negotiation between text in tiny scrollable windows and its questions, alone, is awkward & time-consuming. To attempt a set of compare/ contrast questions while negotiating among 3 screens is impractical.
The author also decries ‘finite detail-oriented questions, multi-step cognitive analysis, archaic texts– These do not seem unreasonable & need examples for support.
.The writer’s most thought-provoking point: in our efforts to reduce curriculum content to testable bytes, are we honing in on one particular type of brain activity, & thus possibly harming students by restricting the path of neural development?
I can only speak confidently to my own field (lit): the same thing has occurred to me in following current ed-policy trends and their effects on ELA teaching. Grammar and editing are left-brained, but creative writing, as well as the synthesis required to, e.g., compare & contrast two text passages– or to draw modern lessons from archaic texts– are right-brained. The exercise of standardizing the teaching of ELA– at least as evidenced by CCSS-ELA– is clearly an attempt to apply measurable formulae to creative thinking. Presumably to make such learning measurable by automatic means. But the result narrows in its conversion to left-brain thinking.
This is the agenda that underlies Coleman’s CCSS-ELA. Purportedly to correct the excesses of self-referential creative writing, it draws on an obsolete lit-crit theory of the mid-20thc. [New Criticism]– which itself was a pendulum-swing away from 19thc. German expressionist lit-crit. To my mind, there’s no question the author Songer hits the nail on the head regarding CCSS-ELA & its assessments. If implementation follows the stds closely, it will squelch any sort of creative thinking– the sort of thinking where one draws in & synthesizes personal experience & background knowledge– in favor of such ‘close-text-reading’ memes as ‘author intent & how achieved via certain words/ phrases’ (etc ad naus).
I’ve had similar qualms about trends in math teaching. Elem math since the early-’90’s (when my kids started) has been engaged in an experiment: attempting to re-do math teaching to suit the artsy-types. I read that CCSS-math in elementary continues the Chicago School ‘Everyday Math’ of the ’90’s. There is a laudable attempt to replace rote learning with concept-learning. But I can’t see it helping much. It manages to hold back the innately-math-advanced, while simultaneously inundating early grades with demands for written concepts just as they’re barely learning to write sentences. (Where is the pedagogy that inculcates number concept for all in PreK/ K, like abacus & manipulables?)
Seems to me it’s the same problem writer Songer describes: instead of making full use of both brain-sides– which everyone uses in concert to some degree– ed-policy (in its search for a simple, testable silver bullet) puts all the weight to one side, thus, no doubt as he claims– favoring one set of neural connections over another, never broadening the curriculum to allow everyone to stretch their limits and come at the concept with everything they’ve got.
Assessments define the curriculum. We need to push assessments to their broadest, most expensive limits. They need to be graded by humans, so we can assess the degree of creativity, grasp of concepts, ability to convey those concepts to various markets (reading audience, tech start-ups, you name it), because that ability is valued in the marketplace of ideas.
LikeLike
bethree5, ok, you have some valid points.
1. Yes, obviously the restriction of test questions to what is included in the passages themselves is not optimal. However, one reason for this policy is to prevent outside knowledge from significantly biasing the tests. If kids’ outside experiences determine their scores (a la college admission essays), then the test is patently unfair.
2. One reason they focus on limited texts is because this is a commonly used skill. Let’s say you are reading the terms of phone plan. Or reading a section of law. Or reading specific computer code documentation. While it helps to understand the context, it often is critical to understand the words and phrases in that one passage. If you cannot, then you can’t understand them in context either. The focus is clearly being placed on career skills over personal enjoyment and edification. That is one of the major goals of K-12 schools. The right-brained activities are hard to measure accurately. And even if they were, are you suggesting that teachers can greatly influence the creativity of students?
3. Note that CC only applies to reading/writing/math. It does not claim to measure everything. Teachers outside these core standards should not be measured on the CC tests. Creative writing would certainly be outside of basic reading and writing. Most would encourage schools to not only offer classes in those advanced skills (creative writing) but to actually measure those more advanced skills. The point is that there must be a bare minimum that K-12 should teach all kids to be functional in their career. Currently, we are not providing those minimum skills.
4. I’m not familiar with “everyday math” but conceptual math is key. Take addition and subtraction. Teachers like to teach the formal “associative rule” (addition doesn’t depend on the order whereas subtraction does). But instead of teaching that rule by memorization, we can teach it conceptually. Addition is combining two groups (two lines into one, two rectangles into one shape, etc.). If you add block A + block B, it doesn’t matter the order. However, if you take block A from inside block B (or vice versa), the order definitely matters. That’s an example of teaching concepts. When kids understand addition is just adding shapes and multiplication is just making a rectangle out of two lines, they understand most of the rules without memorization. Common Core standards are based on these very principles.
I will concede that there are valuable skills you cannot measure on these tests (however, most of these higher level thinking skills are correlated to gains on the standardized tests). But I would caution you to assume that the better teachers are not teaching those skills. Remember that the Gates study showed the best teachers were not deemed to have ‘taught to the test”. Let’s teach all of the above and have alternative measurements in addition.
LikeLike
You are extremely naive if you think the tests aren’t already extremely biased by a student’s outside experiences. Congratulations on your rich white penis.
LikeLike
Such interesting (and completely inaccurate) speculation once again Dienne.
Let’s see. I…
1. Grew up in suburban (most say rural) SC. My parents’ home is now valuedat about $120k. That was my home for my entire childhood.
2. I traveled on planes exactly twice before my senior year. Neither was with my family but rather for academic competition.
3. I had never been to Disneyworld in Mr childhood. In fact, our vacations consisted of visiting my grandparents for a week in the summer.
4. I never traveled overseas until I joined the Navy.
5. I spent more time in the basketball gym than I did reading books. That’s what wet do in Rock Hill, SC. That’s how it produces #1 NFL draft picks like JaDaveon Clowney.
So why don’t you take your specious arguments back to your socialist enclave and get a clue. The whole point of CC and VAMs is to ensure disadvantaged kids have a shot. As Buffet put it, we are not just trying to help those that won the “sperm lottery” (aptitude) but maximize everyone’s potential. Despite your best efforts, we will prevent ineffective teachers from reveling in their complete lack of accountability.
LikeLike