Not everyone who scores Pearson tests is hired from Craig’s List or Kelly Temps. Julie Campbell, a fifth-grade teacher in Néw York recently scored student responses. She stresses that she is not opposed to Common Core or to standardized tests, but she is very troubled by the kind of thinking that is rewarded in the tests.
Because she signed a confidentiality agreement, she does not discuss items on this year’s exams, but released questions from last year.
She writes:
“First things first, one of the most disturbing trends that I have found examining this year’s and last year’s (released) tests is a shift in thinking toward a kind of intellectual relativism. In other words, any claim that a student makes is correct if he or she substantiates it with some evidence. On the surface this doesn’t sound terribly problematic, but when you start to examine some of the anchor papers, the dilemma with this vein of thinking becomes shockingly apparent. The truth is, not all claims are correct and not all evidence is created equal. Making a feeble claim and using evidence out of context to support that claim is an all too common occurrence on these tests….
“According to Pearson “you choose what you think is right” is the first inference. The list of upsides and downsides is one detail. The student then uses an unrelated second detail about joining clubs and school and makes a second inference that you may really end up enjoying it. Formulaically speaking: inference + 2 details will always yield a correct answer[2]. What we have here is a confusing and clumsy answer to a confusing and clumsy question.
“One might argue that this way of scoring allows students to scrape up extra points and is actually a boon to teachers and students alike. It boosts scores! Hurrah!
“But in fact, it creates a terrifyingly slippery slope. I think about climate change deniers, the Creationist Museum in Kentucky that shows humans and dinosaurs roaming Earth side-by-side, 9-11 conspiracy theorists, and the Holocaust itself! Throughout history, people have made misguided claims and have supported their thinking with spurious details and evidence. Don’t our children deserve better?
“Another disturbing pattern that emerges as one reads the anchor responses for the ELA is what I call “The Easter Egg Hunt.” When it comes to short answer questions in particular, the question that is actually being posed rarely matches the answer required. The wordier the written response, the more likely it is that the student will stumble upon the correct answer, find the decorative egg. (Strategy!) Time after time there is a clandestine condition that must be met in order for an answer to get full credit – “Magic Words.” As my scoring instructor illustrated, it’s kind of like tossing all of the words into a bucket and looking for certain key phrases or ideas to float up to the top.”
The nitty-gritty of the scoring process demonstrates that we have outsourced the most important functions of education to a mega-corporation that is incapable of assessing critical thinking. No standardized test can,no matter who writes it or scores it. Standardization itself is antithetical to the intended result.

Knowing what she knows, she’s *still* not opposed to standardized tests?
LikeLike
Perhaps that should be the new definition of brain dead.
“It is difficult to get a man to understand something when his salary depends upon his not understanding it.” Upton Sinclair
What was true then is still true today!
LikeLike
On the relativism side, you may recall that this exact thing happened in the Rialto School District in California, where Holocaust denial “proof” was given to students to write a supposedly “critical thinking” essay on if the Holocaust “actually happened.” A newspaper in the area got the essays through the Freedom of Information Act and found that quite a few students agreed that the Holocaust “never happened,” using “proof” from those appalling “documents.” It’s disgusting.
http://www.sbsun.com/social-affairs/20140711/exclusive-holocaust-denied-by-students-in-rialto-school-assignment
LikeLike
The best examples of “intellectual relativism” are the tests and standards themselves.
They are not based on science (research on teaching and learning) but on uninformed opinion (of Gates, Coleman, Duncan and others) about what standards and assessment should be.
Once you recognize that, you understand that focusing on details (eg, of the tests) is not just a wasted effort but actually a fool’s errand.
LikeLike
EXACTLY!!!!
The fuzzy “thinking” of the whole reform movement.
LikeLike
“. . . but actually a fool’s errand.”
Doing the wrong thing righter, eh, SDP?!?!?! And in the process becoming “wronger”.
LikeLike
Intellectual relative(ity)
You have yours, I have mine
I sell the tests, the pay is fine
Your mind may rot, but hey, oh well
I have the dough so: go to xxxx
LikeLike
I like it!
“Intellectual Relativism”
A common Core for thee
Is Relativity
It’s relatively free
Of value, can’t you see?
LikeLike
But Common Core can be
A load of bucks for free
So valueless to thee
Is valuable to me
LikeLike
Shhhhh. Don’t say this too loud. The alternative is not getting rid of standardized tests; it’s having computers grade the written responses.
LikeLike
My sarcasmometer doesn’t appear to be working this morning. I guess I’ll have to take it in for a 60 year recalibration!
LikeLike
Reblogged this on donotmalignme.
LikeLike
A test/de-facto curriculum that favors the notion that any opinion is valid, as long as it can be loosely supported with “evidence” and “details”, no matter how big a stretch required, seems to be the antithesis of critical thinking. Another opportunity to debunk the edu-fakers bogus claims.And further proof that Common Core ELA standards shun content in favor of abstract skills that are essentially meaningless.
LikeLike
Of course this whole process works very well in a world where context is unimportant, where close reading is supposed to reveal the author’s intentions, right or wrong, as truth.
LikeLike
My experience several years back scoring BCR’s and ECR’s was similar to this writers experience. I must say: “The truth is, not all claims are correct and not all evidence is created equal. Making a feeble claim and using evidence out of context to support that claim is an all too common occurrence on these tests….” These kids are going to make today’s politicians look like intellectuals. Yikes!
LikeLike
I read the whole article. It should be nailed to the door of every educational organisation, and gates can insist that it become the default screen saver for MSWindows (when ????????)
LikeLike
. “Making a feeble claim and using evidence out of context to support that claim”
SOP for ed-reformers
LikeLike
They can only evaluate what they know. They are doing what they do best.
LikeLike
Anything Goes:
Written English has been replaced by texting.
De-valuing evidence is the next thing.
LikeLike
I also scored the NY 6-8 grade ELA tests and can attest that what the writer said is true. The sad part is, the student can make any claim and throw out a bunch of direct quotes from the text and get full credit. We called this the throwing everything against the wall approach. The student can literally write one original word in a short constructed response and receive full credit. What’s even sadder is that punctuation doesn’t count. Paragraphs and periods don’t matter. If parents knew just how poorly their child who “passed” really wrote, they would be up in arms.
LikeLike
So have we substituted a claim and warrants formula for the five paragraph essay, or does the process get plugged into the five paragraph format? If I ever have to take another test, I want to make sure to feed them the correct template. 🙂
LikeLike
” If parents knew just how poorly their child who “passed” really wrote, they would be up in arms.”
NO!, they shouldn’t be up in arms over that aspect as that “passing” means absolutely nothing considering the whole process of educational standards and standardized testing is COMPLETELY INVALID and ANY RESULTS therefore are, according to Noel Wilson, “VAIN AND ILLUSORY”.
To understand why this is so please read Wilson’s never refuted nor rebutted destruction of the two most nefarious educational malpractices used these days, educational standards and standardized testing. In the MOST IMPORTANT EDUCATIONAL STUDY OF THE LAST HALF CENTURY Wilson shows the fundamental epistemological and ontological errors and falsehoods involved in those malpractices that render any results COMPLETELY INVALID. See his: “Educational Standards and the Problem of Error” found at: http://epaa.asu.edu/ojs/article/view/577/700
Brief outline of Wilson’s “Educational Standards and the Problem of Error” and some comments of mine.
1. A description of a quality can only be partially quantified. Quantity is almost always a very small aspect of quality. It is illogical to judge/assess a whole category only by a part of the whole. The assessment is, by definition, lacking in the sense that “assessments are always of multidimensional qualities. To quantify them as unidimensional quantities (numbers or grades) is to perpetuate a fundamental logical error” (per Wilson). The teaching and learning process falls in the logical realm of aesthetics/qualities of human interactions. In attempting to quantify educational standards and standardized testing the descriptive information about said interactions is inadequate, insufficient and inferior to the point of invalidity and unacceptability.
2. A major epistemological mistake is that we attach, with great importance, the “score” of the student, not only onto the student but also, by extension, the teacher, school and district. Any description of a testing event is only a description of an interaction, that of the student and the testing device at a given time and place. The only correct logical thing that we can attempt to do is to describe that interaction (how accurately or not is a whole other story). That description cannot, by logical thought, be “assigned/attached” to the student as it cannot be a description of the student but the interaction. And this error is probably one of the most egregious “errors” that occur with standardized testing (and even the “grading” of students by a teacher).
3. Wilson identifies four “frames of reference” each with distinct assumptions (epistemological basis) about the assessment process from which the “assessor” views the interactions of the teaching and learning process: the Judge (think college professor who “knows” the students capabilities and grades them accordingly), the General Frame-think standardized testing that claims to have a “scientific” basis, the Specific Frame-think of learning by objective like computer based learning, getting a correct answer before moving on to the next screen, and the Responsive Frame-think of an apprenticeship in a trade or a medical residency program where the learner interacts with the “teacher” with constant feedback. Each category has its own sources of error and more error in the process is caused when the assessor confuses and conflates the categories.
4. Wilson elucidates the notion of “error”: “Error is predicated on a notion of perfection; to allocate error is to imply what is without error; to know error it is necessary to determine what is true. And what is true is determined by what we define as true, theoretically by the assumptions of our epistemology, practically by the events and non-events, the discourses and silences, the world of surfaces and their interactions and interpretations; in short, the practices that permeate the field. . . Error is the uncertainty dimension of the statement; error is the band within which chaos reigns, in which anything can happen. Error comprises all of those eventful circumstances which make the assessment statement less than perfectly precise, the measure less than perfectly accurate, the rank order less than perfectly stable, the standard and its measurement less than absolute, and the communication of its truth less than impeccable.”
In other word all the logical errors involved in the process render any conclusions invalid.
5. The test makers/psychometricians, through all sorts of mathematical machinations attempt to “prove” that these tests (based on standards) are valid-errorless or supposedly at least with minimal error [they aren’t]. Wilson turns the concept of validity on its head and focuses on just how invalid the machinations and the test and results are. He is an advocate for the test taker not the test maker. In doing so he identifies thirteen sources of “error”, any one of which renders the test making/giving/disseminating of results invalid. And a basic logical premise is that once something is shown to be invalid it is just that, invalid, and no amount of “fudging” by the psychometricians/test makers can alleviate that invalidity.
6. Having shown the invalidity, and therefore the unreliability, of the whole process Wilson concludes, rightly so, that any result/information gleaned from the process is “vain and illusory”. In other words start with an invalidity, end with an invalidity (except by sheer chance every once in a while, like a blind and anosmic squirrel who finds the occasional acorn, a result may be “true”) or to put in more mundane terms crap in-crap out.
7. And so what does this all mean? I’ll let Wilson have the second to last word: “So what does a test measure in our world? It measures what the person with the power to pay for the test says it measures. And the person who sets the test will name the test what the person who pays for the test wants the test to be named.”
In other words it attempts to measure “’something’ and we can specify some of the ‘errors’ in that ‘something’ but still don’t know [precisely] what the ‘something’ is.” The whole process harms many students as the social rewards for some are not available to others who “don’t make the grade (sic)” Should American public education have the function of sorting and separating students so that some may receive greater benefits than others, especially considering that the sorting and separating devices, educational standards and standardized testing, are so flawed not only in concept but in execution?
My answer is NO!!!!!
One final note with Wilson channeling Foucault and his concept of subjectivization:
“So the mark [grade/test score] becomes part of the story about yourself and with sufficient repetitions becomes true: true because those who know, those in authority, say it is true; true because the society in which you live legitimates this authority; true because your cultural habitus makes it difficult for you to perceive, conceive and integrate those aspects of your experience that contradict the story; true because in acting out your story, which now includes the mark and its meaning, the social truth that created it is confirmed; true because if your mark is high you are consistently rewarded, so that your voice becomes a voice of authority in the power-knowledge discourses that reproduce the structure that helped to produce you; true because if your mark is low your voice becomes muted and confirms your lower position in the social hierarchy; true finally because that success or failure confirms that mark that implicitly predicted the now self-evident consequences. And so the circle is complete.”
In other words students “internalize” what those “marks” (grades/test scores) mean, and since the vast majority of the students have not developed the mental skills to counteract what the “authorities” say, they accept as “natural and normal” that “story/description” of them. Although paradoxical in a sense, the “I’m an “A” student” is almost as harmful as “I’m an ‘F’ student” in hindering students becoming independent, critical and free thinkers. And having independent, critical and free thinkers is a threat to the current socio-economic structure of society.
LikeLike
Great post! Thank you.
LikeLike
I think the big takeaway of this discussion is this:
When it comes to writing, the backers of high-stakes testing don’t really care what our kids have to say–it’s irrelevant to them. They just want to promote a system that can produce a data point for power and control and for privatizing our public schools and profiting from this whole process. The idea of kids as authentic writers and communicators is just not important to them.
LikeLike
100%
The backers of high stakes testing care about student reading/writing (or math skills) as much as a snake oil salesman cares about his customers health.
LikeLike
Reblogged this on VAS Blog and commented:
The dumbing down of testing – abysmal quality control could be heading our way with Pearson Education’s now involved in NAPLAN
LikeLike
For those of us who are diagnosed AI, what is NAPLAN?
LikeLike
Googling works:
http://www.nap.edu.au/naplan/naplan.html
LikeLike
Sounds like NAPALM for the teaching and learning process!
I guess they don’t pay attention to their native son Noel Wilson in Australia either. Ay, ay, ay!!
LikeLike
Time for someone to create an acronym website.
It could be titled, “A.R.U.”
LikeLike
After reading this posting and being reminded of how CCSS and its high-stakes standardized testing are conjoined twins, read another one from this blog entitled “Common Core for Commoners, Not My School!”—
[entire posting start]
This is an unintentionally hilarious story about Common Core in Tennessee. Dr. Candace McQueen has been dean of Lipscomb College’s school of education and also
the state’s’s chief cheerleader for Common Core. However, she was named headmistress of private Lipscomb Academy, and guess what? She will not have the school adopt the Common Core! Go figure.
[entire posting end]
Link: https://dianeravitch.net/2014/03/23/common-core-for-commoners-not-my-school/
When it comes to the leading rheephormsters—
For OTHER PEOPLE’S CHILDREN aka the vast majority, standardized mediocrity and dumbing down, all in the service of keeping them in their place. *Those whom Rahm Emanuel calls the “uneducables” and Michael J Petrilli the “non-strivers.”*
For THEIR OWN CHILDREN aka the putatively meritorious and deserving, genuine learning and teaching, all in the service of maintaining the privileges and advantages to which they have been born.
So when it comes to the 2014 posting I reference above, the modifier “hilarious” should be understood as meaning “the joke is on us.”
In the most excruciatingly painful ways imaginable.
Just my dos centavitos worth…
😎
LikeLike