In this mini-essay, left as a comment, Bob Shepherd notes that Common Core testing assumes that there is only one correct answer when interpreting literature. This, he says, is a complete rejection of reader-response theory, which had been prevalent for many years. Shepherd has many years of experience writing curriculum, assessments, and textbooks.
He writes:
“Years ago, I was doing a project for one of the major textbook publishers—writing for a high-school British literature textbook. I was given an assignment to write a lesson on Robert Burns’s poem “A Red, Red Rose.” This poem begins, you may remember, with the following lines:
O my Luve’s like a red, red rose
That’s newly sprung in June;
O my Luve’s like the melodie
That’s sweetly play’d in tune.
One of the questions that I asked about the poem was, “Why does the speaker compare his beloved to a red, red rose?” And the answer I wrote for the answer key was something like, “The speaker wishes to communicate that his this person is attractive and that he loves her, and so he compares her to a red rose, which is a traditional symbol of beauty and of romantic love.” I could have elaborated: Probably through association with blood and with blushing, the color red traditionally symbolizes intense emotion, or ardor. Roses are attractive and share this property with the objects of romantic affection. For these reasons, it became conventional to speak of someone as being “a red rose” in order to communicate that a) she (or, more rarely, he) was beautiful and b) that she (or, more rarely, he) was an object of ardent emotion, and c) that that emotion was one of romantic attraction. The speaker is therefore using a conventional symbol.
I could have added that the reason why the poet chose to express this in a simile rather than in a metaphor (“O my Luve’s a red, red rose”) was probably as mundane as to fill out the meter. I could further have explained that it is the beloved not the speaker’s feeling that is compared to a rose, for later in the poem, the speaker uses the same word, Luve, in direct address:
And fare thee well, my only Luve
And fare thee well, a while!
And I will come again, my Luve,
Tho’ it were ten thousand mile.
The editor wrote back to me saying, “Don’t presume to tell students that there is ONE CORRECT interpretation of the line.” I responded, “What should I say instead?” She wrote back, “Say, ‘One possible reason is that red roses are traditional symbols of beauty and of romantic love.’” I pointed out that if I were to follow her advice, I would have to include a similar disclaimer (“one possible”) in almost EVERY STATEMENT made about any work of literature in the book, which would make for awkwardness. She informed me that I was being overly directive and not respecting the students’ right to his or her own interpretation and that this made her question my suitability for the job she had asked me to do.
Let me hasten to add that I do understand where that editor was coming from. She held a version of a reader response theory of literature that goes something like this: a text means whatever the reader constructs when reading it. This grotesque misunderstanding of what “a reader’s construction of a text” can reasonably mean had become the de facto orthodoxy in ELA lit texts at the middle-school and secondary-school levels. I call this a grotesque misunderstanding because a text is an act of communication and as such depends, usually, upon shared usages and upon the belief on the part of the reader and the writer that communication across an ontological gap of a communicable meaning is possible. To deny that—to say that any text can mean anything—is to undercut the very notion of communication, of transmission across that gap from one subjectivity to another. Part of teaching people how to read literature is to teach them about conventional usages and what those can reliably be taken to mean.
Now, one might say, but wouldn’t an alternate reading like the following be acceptable?
The convention of the red, red rose as a symbol of feminine beauty is part of an complex of objectifications found in poetry and song produced by men, particularly in the Cavalier and early Romantic periods, and the speaker probably uses this because he is a conventional, unthinking, objectifying pig.
The editor might have had a student response like that in mind.
But here’s the problem with that: the editor would be confusing significance (meaning as mattering to the reader) with interpretation (meaning as the intent of the author). Failure to observe this distinction leads to a lot of complete nonsense in writing and speaking about literary texts. The differing responses are to differing matters–what the author intended and what significance what the author did has for a particular reader.
So, how does all of this relate to the new tests?
Well, one remarkable characteristic of the new tests is that they have COMPLETELY OVERTHROWN what was the STANDARD CHURCH ORTHODOXY in K-12 ELA–the prevailing Reader Response/Constructivist/The Author Is Dead orthodoxy that texts have alternate readings, constructed by readers, that have to be respected. For the most part, the questions about literature on the new exams assume that THERE IS ONE CORRECT ANSWER. Am I the only one to notice that? Did millions of English teachers and textbook writers who were of the “readers construct texts” or “reader response” schools of thought suddenly change their minds about this?
No, their minds were changed for them, de facto, by people constructing the new tests based upon the new standards.
Shouldn’t I be pleased about this, given my defense, above, of the “one true” reading of the line from Burns? No, and here’s why: What we mean by “What does this mean?” itself differs depending upon whether we are talking about intent or significance, and intent itself is by no means cut-and-dried, simply there for discovery. Getting at intent involves a great deal of knowledge of matters like literary conventions and genres and techniques and historical periods and the thought and practice and life experience of the author and much, much else. So even if we made the assumption that any question on a standardized test must deal with intent and not with significance, it would still be the case that particular passages would be open to varying interpretations.
And the relevance of extra-textual matters to interpretation raises another issue with regard to the approach to literature instantiated in the new standards. Students and teachers are encouraged in these to follow a New Critical procedure—to examine closely the text itself, without reference to external materials. But intent does not exist in a vacuum. If someone leaves a note saying, “Tie up the loose ends,” it matters whether the note is from a macramé instructor or from a mob boss worried about possible informants! Texts exist in context.
When I examine the new tests and the questions asked on them, my overwhelming impression is that the questions were written by people who hadn’t the subtlety to understand what a complex business learning to read carefully and well is. As often as not, the questions FAIL because the question writer did not himself or herself understand some subtlety. Let me give an example to illustrate what I mean.
Suppose that a question on one of these tests reads as follows:
Which of the following best describes the attitude of the speaker in the first line of Burns’s “A Red, Red Rose”?
A. Ardent affection
B. Casual interest
The test writer would probably think that answer A. is the correct answer.
But consider this: A deconstruction of that first line would look beyond the verbal intention—the intended communication—to other factors getting at significance. Why did the speaker use a hackneyed, conventional expression? Why did he express the conventional association in a simile instead of in some more sophisticated way? Do the facts that he chose a hackneyed convention and chose the simile, most likely, simply as an easy way to fill out the meter suggest that he did not give this poem much work or thought? In other words, is this first line suggestive of someone who is not as serious as would be another poet who, in this circumstance, would bother to say something original and real? I’m reminded of a fellow I knew when I was a kid who had written what he called a “general purpose love song.” He said to me, “The beauty of this song is that I can throw any girl’s name in there. Miranda. Amanda. Sweet, sweet Jane.” Is the casualness of Burns’s line related to the fact that in the last stanza, he’s outta here?
“Hey, you’re great. Really. I’ll be thinking about ya. Outta here.”
Hmm. Suddenly the wrong answer starts to look as though it might not be so wrong after all because now we are talking not about intent but about significance. Is this an accurate reading of the significance? I love Robbie Burns. I have participated with great delight in Burns dinners (though I shall always pass on the haggis). But he was a notorious womanizer, and this poem is a piece of tossed-off minstrelsy and not a great work of art like his “Song Composed in August” or “To a Mouse.” I don’t mean to take away from the poem by saying that. It’s a perfect specimen of its type. But it’s a conventional type. It’s a “My Darlin’ Clementine,” not Yeats’s “The Folly of Being Comforted” or Millay’s “Love Is Not All” or Burns’s own “John Anderson, My Jo.”

“The Right Ones” (TROs) know life has “Only One Right Answer” (OORA❢ OORA❢).
“The Rest Of Us” (TROU) know it bears so many more answers “Left Over” (LO❢).
LikeLike
I’ve been an 8th grade English teacher for more than 17 years, and I learned a lot from Mr. Shepherd’s mini-essay. This is so thought provoking. Thank you for posting it.
LikeLike
I think the biggest absurdity of the common core test lies in the fact that it attempts to assess depth of knowledge 4, which is obviously geared towards major projects like term papers, design projects, etc. Assignments like these require months of time to gather and study multiple sources of information and can only truly be assessed by the student’s teacher. These types of projects are great for the diligent “work-horse” student who may not be so adept at passing a standardized test and it’s sickens me to consider the possibility that type of student might be shown to be “not proficient” in DOK4 when in reality it might be an area of strength for them.
LikeLike
amen to this
LikeLike
We often had students who got this type of question wrong because they “overthought” the response. The answer was the “no brainer” response, not meant to be analyzed. Yet, when asked, the student would have compelling and even irrefutable evidence in support of their choice.
It all boils down to outcomes and expectations. Do we want generic outcomes or critical thinkers. Smart kids learn how to play the game.
LikeLike
DOK = POS
LikeLike
In the bit switch, binary universe of computers , there are only two states of being: 1 or 0, on or off, correct or incorrect. Of course, expert systems (“artificial intelligence”) have made incredible strides and computers can now mimic human behavior in amazing ways, playing chess, choosing stocks, diagnosing illness. But, right now, at the root level, they are still all about very complicated arrangements of lots and lots of 1s and 0s. Teaching, like interpreting and discussing a poem (to use Bob’s example) indeed requires subtle shades of human understanding, communication and behavior. When it comes to our schools what parent would want their child’s first grade teacher to be a competent instructor but to NOT love children, too? But to reformers like Bill Gates those very human, messy qualities like love can’t be quantified so do they are being left out of their equations. The history of science and technology has shown that If a machine cannot be made (yet) to do a human’s job, then humans will be made to act like machines. It’s tragic, really, Bill Gates might be the most famous computer scientist in the world but for my money the late Joseph Weizenbaum was the wisest. I was lucky enough to meet Weizenbaum twice. His book, Computer Power and Human Reason (1976) foreshadowed some of the difficult choices that we are being challenged by today.
LikeLike
Ultimately, we need to get rid of the notion that students are commodities. Somehow, humanity has been left out of the equation.
LikeLike
Thank you, Dr. Ravitch, for posting this. This was a comment on a thread. Had it been a completed essay, I would have drawn explicitly, the ineluctable conclusion from the observations above:
The new tests are based on fundamental misunderstandings of the reading of literature and of the arts of interpretation. Because of the test makers’ misunderstandings, the mostly “objective format” test questions are COMMONLY themselves misconceived. Often, arguably, for a given question, a) more than one of the answers is correct, b) none of the answers is correct, or c) the question is not answerable as written.
This is the sort of thing that happens when amateurs venture into areas they know nothing about. I ook at these tests and think that this is what a licensing exam for physicians would look like if I, who have no training in medicine, wrote one. The new national tests are embarrassingly bad, and it is a national scandal that anyone is taking them at all seriously.
The test makers dare not release their test questions because those questions would not hold up under scrutiny by experienced teachers who actually know what they are doing–by teachers who have themselves learned something of the arts of hermeneutics and who teach those arts in their classes.
The tests are the wrong tool for the job of figuring out whether kids can read literature well. One might as well try to measure the width of a synapse with a yardstick or do technical drawing with crayons.
LikeLike
It’s really a national scandal. The new Common Core tests in ELA are a scam.
LikeLike
And it’s time for English teachers who actually know what they are doing to start calling out the charlatans who are perpetrating the scam.
LikeLike
What about the new SAT? I’m concerned about children having to determine the meaning of the word in the context of a passage, as the author meant it. Theoretically, a student could be forced to choose the meaning, as they think the author meant it to mean, and not what the word may actually mean. This seems like a mind-reading exercise, and totally subjective. Right?
LikeLike
Reading IS mind reading. What else could it be? We do well to bear that in mind–that reading is fraught with complication, as all mind reading is. That’s something that the test makers do not understand. But mind reading is not entirely subjective. Heck, over a hundred years ago, Darwin wrote a book about how similar facial expressions and gestures were across the animal kingdom. We read them. That’s mind reading. It’s based on shared, verifiable, objective observations, but it’s about subjective phenomena, and it’s contextual, and these are complications.
LikeLike
This is so freaking well-put, Bob.
LikeLike
Thank you so much for these posts. They are brilliant; they are also heartbreaking. I might add that the CCSS-based tests do the same thing with respect to writing.
LikeLike
I argued this when I was a student in AP English (1990) and we had to “correctly” analyze prose and poetry in preparation for the test. How do we know what the author really “thought,” I would ask my teacher (she was a saint to put up with me), unless we dig the person up and ask him or her? I did learn for the AP test that if the author was listed (authors of the pieces usually weren’t given), I should analyze the opposite of what I should expect the author to normally say. On the test, there was a poem by Emily Dickinson on death, and I said that in that poem that Emily Dickinson thought death was great. I must have done it right, since I got a 5 on the test, but I’ve always thought that was a stupid way to look at poetry and prose.
LikeLike
How do we know the author’s intent?
Well, there are lots and lots of clues.
First, there are all the conventions of language–conventional definitions and usages, for example–and of genre. If you are reading an Elizabethan drama, and a court fool takes the stage, you can expect that, by convention, the fool will speak unconventional truths–that the fool will dare to say, under cover of a jest, what others think but dare not say.
Second, there is the body of work of the author. If you think that this particular poem by Wallace Stevens is about taking solace in, comfort in, pleasure from, present sensory experience, rather than in spiritual experience, you are probably right, for this is an idea to which Stevens returns again and again and again.
Third, there are the understandings available in the author’s time and place. If you think that Little Red Riding Hood is an allegory about capitalist exploitation of workers, you are simply wrong because Capitalism as we understand it didn’t exist when this story was created.
LikeLike
The point is that there are different kinds of meaning. Intent is not the same as significance to the reader. The latter will be enormously varied and unpredictable. The former will require a great deal of reflection and research, and different readers, having done different reflection and research, will come to differing conclusions (and these will be colored by the interpretive communities of which they are part).
In short, none of this is cut and dried. And that’s the point. A saying often attributed to Einstein goes, “We must make things as simple as possible but not simpler.”
Reading well is not simple. The tests are, and so they fail.
LikeLike
Of course there are ways to tell. But how does one “know” that George Bernard Shaw wrote a piece that is NOT satirical? Or that Emily Dickinson wrote a poem welcoming death? That’s what I mean. Those AP questions, at least then, were designed to trick students. That’s not the way it should be.
LikeLike
By the way, I wrote an entire paper in an Honors History class on Little Red Riding Hood (the old original one) that argued that the wolf was mocking the Eucharist.
LikeLike
lol. See Paul Frye’s Theory of Literature and his many amusing readings of “Tony the Tow Truck.” And, of course, there are the wonderful satirical books of critical interpretations of the Winnie the Pooh stories–The Pooh Complex and The Postmodern Pooh, if memory serves me well.
LikeLike
Kind of makes me think of Back to School when Rodney Dangerfield’s character hires Kurt Vonnegut to analyze his own writing for a lit course paper, and the paper gets an F, the professor saying, “Whoever wrote this doesn’t know the first thing about Kurt Vonnegut!”
LikeLike
LOL. I suspect that Vonnegut, who is in heaven now, would have agreed that he was no expert on himself, given the dubiousness of the notion of the Self. His work, of course, is pretty didactic, and its general intent is rarely much of a mystery, but intent is not all.
I’ve always enjoyed, immensely, Vonnegut’s occasional writing and speech about writing and literature. He once wrote about why he gave up teaching literature. He had an epiphany one day when he was standing in front of a class, preparing to explain “The Dead,” from Joyce’s Dubliners, to his students. He was suddenly sickened at the prospect of reducing a complex, living work to a few facile generalizations. Whatever the critical approach we take, we do well to keep that in mind. Great works ramify. They have a way of becoming worlds. Wallace Stevens wrote that a great poem just escapes the understanding. Vonnegut would have approved of that, I think. English teachers often play this game of reducing works to facile generalizations: e.g., Macbeth is “about” how evil rebounds upon its perpetrator. Instead of having our students draw general conclusions, we should often, I think, have them specify. Well, given what we know, how would the snow have looked on this day? the sky? What would Macbeth have been carrying on his person? What might he have been hearing or smelling at this moment? What concerns might he be entertaining? Just as the method actor is encouraged to know what’s in his or her character’s pockets and what the character likes to eat for lunch, so a reader of a great play or novel or poem can often fairly fully imagine a time and place based on the clues the work provides. One of the distinguishing features of great literary writing is this SPECIFICITY via suggestion. Stevens, again:
I do not know which to prefer,
The beauty of inflections
Or the beauty of innuendoes,
The blackbird whistling
Or just after.
We should probably be giving a lot more attention than we do to training students in how to enter into the world of a work in all its specificity–training in how to imagine a work fully while reading. Ideally, the student enters into the world of a work and has an experience there, and it is that experience that has meaning. This view of literature is fully compatible with an intentionalist stance, for writers intend to create particular worlds so that readers might have particular experiences there.
LikeLike
The problem is that if several answers are correct, and only 1 is counted, then its about trickery. Aren’t these new tests made to be vague and convoluted to promote choosing the wrong answer among several correct answers among the bubbles?
As to essays, aren’t “they” trying to get computer programs also to grade essays, which on a prior entry it was shown how a complete hogwash nonsense essay, if correctly punctuated, could pass the computer essay grading?
There should be one clear answer. As with math, there is only one answer, because math does not lie, though statistics can be bent and tortured.
With language, the answers should be clear, and there should only be one real answer, not trickery.
LikeLike
The test makers don’t recognize that their questions are TYPICALLY so absurdly framed that they have no correct answer or multiple correct answers or are unanswerable. I often read these tests and have an overwhelming desire to enroll the test authors in an elementary course in literary interpretation. BTW, one of the criteria given to test writers is that the “distractors”–the wrong answers–must be “plausible.” Well, what does plausible mean? It means “seemingly or apparently valid, likely, or acceptable; credible.”
Credible but wrong, unlike the one true and correct answer in the black and white world of the test maker.
It’s no great mystery that passing scores on the New York state exam dropped by 30 percent on the PARCC (spell that backward).
Who will guard the guardians? wrote Juvenal.
And who will test the test makers?
It is our duty to do that.
LikeLike
The best students are the ones that have the most difficulty finding the one right answer because they see so many possibilities in the plausible distractors, and sometimes even in the flatout wrong answers. When I give my own tests, I’m much more concerned with the support students give for the answers they choose. Sometimes they actually show me a perspective I hadn’t thought of. I can’t stand even thinking about these tests which are bad and wrong in so many ways. They are a waste of precious time (and money) but they are worse than a waste since decisions affecting peoples’ lives are based on the outcome of these inane tests. We are living and working in an absurd environment.
LikeLike
Diana, this reminds me of a text question on a math test that we saw last year (as ESE teachers, we read the math questions and answers to our students who have that accommodation). This question had two correct answers on it. The question showed line segments that formed a figure and asked what the name of the figure was. The thing was, if you looked at it as a two dimensional figure, it was one thing, and if you looked at it as a three dimensional figure, it was another. Some students immediately saw one, and some immediately saw the other. Only one answer was on the test; the students who could not for the life of them see the figure in 2D because the 3D figure was as plain as day to them were penalized for not seeing as the question designer saw.
LikeLike
There should be only one right answer because everything in life has only one right answer and it’s the one your betters tell you (even if they told you something different yesterday). If Mr. Shepherd thinks otherwise, I’m afraid I’m going to have to report him to the Ministry of Truth.
LikeLike
LOL. I am resigned to this, Dienne!
LikeLike
🙂
LikeLike
This whole episode makes the NCTE look as rotten and sold out as the textbook and testing companies themselves.
LikeLike
I can only imagine what would the AMA would have done had Coleman presumed to write new standards for the practice of medicine based on the 1858 edition of Gray’s Anatomy and if new bullet tests had been based on these. It’s a scandal that the NCTE has not come out clearly and forcefully against these tests, which are obscenely amateurish and entirely misconceived.
LikeLike
The NCTE actually reviewed the standards and issued critiques three times (on their website, but hard to find), even saying they thought the sheer number of early standards might inhibit literacy, and others had no foundation in research. Worse, some standards, they said, went against best practices. However, their silence now implies support for the standards.
LikeLike
Yes, I read one draft of the NCTE review. It seemed to me extraordinarily shallow. In particular, it did not question a lot of fundamental misconceptions underlyng the new “standards.” For example,
it did not address the fact that the language standards seem to have been written in complete obliviousness of what has been learned in the past 50 years about language acquisition and appear almost completely at random
or the fact that the writing standards vary little from level to level, are extraordinarily vague, do not address specific techniques in the toolkit of the writer, and are based upon very dubious notions about modes of writing
or the fact that the literature standards are almost exclusively a list of vague, general statements about formal attributes of literary works; contain almost no content; are full of lacunae; are often completely arbitrary or random; and concentrate, often, on relatively minor matters like the effect of figurative language on tone and mood as opposed to major matters like the metaphorical underpinnings of most conceptualization, literary and otherwise
or the fact that the literature standards instantiate some approaches to literary interpretation that are largely discredited after three quarters of a century of scholarly and critical critique of those approaches.
The new ELA standards look to me like what one would get if one asked a group of random nonliterary persons from some small town Rotary Club to make a list of “stuff to do in English class.” This is not the work of professionals, and the proper response to it would have been derision. The authors of the new ELA “standards” should have been laughed off the national stage by this professional organization, and it’s shocking that that’s not what happened. The new “standards” are amateurish work. If they had been prepared as a class project for an English methods class, they wouldn’t be acceptable for submission. The authors would have to be asked to go back and do some real thinking and research and to redo what they’ve done in light of what we’ve learned about the teaching of English over the past century.
LikeLike
The NCTE actually reviewed the standards and issued critiques three times (on their website, but hard to find), even saying they thought the sheer number of early standards might inhibit literacy, and others had no foundation in research. Worse, some standards, they said, went against best practices. However, their silence now implies support for the standards. *this comment was meant to go here*
LikeLike
ugh! never mind, I give up trying to put it where I meant it to go.
LikeLike
42
LikeLike
lol
LikeLike
Mr. Shepherd, This was such a good piece of thinking and explanation to read that I looked up your blog –
Unfortunately, I’ll probably never have time to read it, due to my demanding job. And I’m trying to do my own writing on the reform movement. It’s moving slowly…
The more voices out there, the better. Have you managed to get your opinions into some relatively mainstream, larger publications? If so, please let us know what. If not, please try, and keep trying.
LikeLike
I can name a phrase that has “one meaning”… “ed reform” … it means “baloney”! Although some may say it means “bull s” or “corporate profiteering” so yes Bob Shephard… I guess you are right!
LikeLike
“ed reform” a misnomer for “edudeform” is “re” backward which stands for “rheeal excrement”. (Had to bring a KTAism in for that one.)
LikeLike
Othello’s tragic flaw is that he is unable to sustain uncertainty. “To be once in doubt. Is to be resolved,” he says. There are thick, Othello-like persons in the world–ones who cannot bear the possibility that meaning might be complicated. But to those folks I would say, the author’s intent is different from the significance to you as a reader. Significance is important. It’s why we read. But it will differ from reader to reader. If I were to take my five-year-old grandson to the Roman forum, I might stand in a place, there, and get all excited about this being the very spot where Marc Anthony gave the oration over Caesar’s body. He might get similarly excited because there are lizards running about. The experience would have a different meaning, in the sense of significance, to us.
But even intent is not simple. Often, it has to be recovered. I first read something by Plato back when I was in junior high school. So, I’ve been reading him, off and on, for many decades. But only recently have I come to understand something about Plato and his time that I think is key to recovering his intent. In Plato’s Greek, the word typically translated into English as “virtue” means something like “efficiency.” So, one could speak of a virtuous man, a virtuous state, or a virtuous shoe, the last being one that lasts a long time, is comfortable, etc.–one that meets predetermined, established, eternal criteria discoverable by reason. This notion, that virtue might be simply THERE to be discovered by an inquiring mind–an absolute, unchanging truth about which there could be no doubt on the part of one who has discovered it–seems to me, to be a great key to understanding how Plato thought–an unexamined assumption on the part of the one who famously, through his character, Socrates, called upon us to live the examined life. This fact about the Greek that Plato spoke has enormous explanatory power. It helps to explain, for example, the notorious totalitarianism that Plato was capable of. Truth was absolute, and preexisting, and there to be discovered by those with sufficient wit and other qualities to do so. It seems to me a delicious irony that the one who wished to live the examined life was incapable of examining this freighted assumption.
I mention this story because it illustrates a few notions about intent. Intent, especially the intent of an author from a very different time and place, has to be recovered, and recovering it is no simple matter. We see through our own lenses, and attempting to see through other lenses, very different from our own, is challenging. That’s why we have kids do research. If a teacher wants a student to understand what’s going on in Emerson’s “Brahma”–what Emerson’s intent was–then he or she is going to have to introduce the student to the ideas from the Upanishads–newly translated in Emerson’s time–that Emerson had been reading.
And that’s usually the case. Texts exist in context, and treating them out of context, as is done on these national tests, is an obscenity that distorts good reading practice as surely as having kids toss off a draft for a writing test distorts good reading practice.
The national standardized tests simply are not appropriate instruments for testing what we teach in English language arts.
LikeLike
That context matters seems self-evident, but apparently reformers have on lenses that filter that out. Good to see you back in the comments (and main posts), Bob!
LikeLike
“It seems to me a delicious irony that the one who wished to live the examined life was incapable of examining this freighted assumption.”
Damned postmodernists always have to ruin a good thing, eh!!-ha ha!!
LikeLike
Oddly, the new standardized Florida math test will have problems in which there are multiple correct answers (not bubble-in), and all of them have to be listed to get the problem correct. In elementary school. I don’t foresee disaster with this at all…
LikeLike
Yep. Welcome to Utah’s CC test, which we are selling to you folks.
LikeLike
Aye, thanks. I imagine the conversation between Florida and Utah went something like this:
Utah: *rings doorbell*
Florida: *opens door* Can I help… sorry, we don’t have any money for anything right now. I’m barely keeping the lights on. Sorry.
*starts to close door, stops suddenly* Wait, what’s that you’re selling?
Utah: It’s a test that will…
Florida: A test? Why didn’t you say so in the first place? Hold on a sec. *gets out wallet* Pleasure doing business with you. Do me a favor and keep me updated on your products, will you?
Utah: Oh, will do, sir, will do!
Florida: Thanks! And here’s a little something for yourself! *slips Utah a twenty*
LikeLike
Bob Shepherd: perhaps a very dead and very old and very Greek guy would come in handy here—
“As a vessel is known by the sound, whether it be cracked or not; so men are proved, by their speeches, whether they be wise or foolish.” [Demosthenes]
Since ya ain’t cracked or foolish….
Kudos.
😜
And let me suggest a slightly different way of looking at this posting and thread. Let’s focus on standardized testing.
Among others, Banesh Hoffman (THE TYRANNY OF TESTING, 2003/1964/1962) and W. Edwards Deming (THE ESSENTIAL DEMING, 2013) pointed to the inherent difficulty with the best standardized tests, those most carefully designed and pre-tested and administered—
They are not concerned with the quality of the thought that went into the answer but, simply, the answer itself. Hence, a “wrong” answer—from the test maker’s POV, supported by innumerable statistical tests and trials—may be the most original and creative and sensitive and sensible way to “answer” the question but that doesn’t literally count for the test taker; rather, it counts against him/her. In other words, the test taker will be penalized for “thinking outside the box”—in this case, the “box” the test makers put the test taker in.
There is no getting around this conundrum. That’s the nature of standardized tests, to the nth degree when we look at the high-stakes kind. *Google: pineapple, hare, Daniel Pinkwater.*
Again, thank you and everyone else for the posting and commentary. And a hearty thank you to the owner of this blog for showing once again in practice what a “better education for all” looks like.
😎
LikeLike
Again, we are looking for standardization, not creativity or even basic thinking skills.
And how does this indicate a child is college or career ready?
If anything, we are now looking at ways to dumb our students down.
LikeLike
Bob, what’s your opinion on this:
http://www.mhecommoncoretoolbox.com/close-reading-and-the-ccss-part-1.html
It’s been over two decades, but I did read Louise Rosenblatt’s book, Literature as Exploration, and it seems to me that Dr. Fisher’s namedrop of Rosenblatt is an exercise of bridging a gap that’s unbridgeable, attempting to put a Reader Response veneer over There’s One Right Answer And It’s In The Text.
LikeLike
I would say that Dr. Fisher needs to go back and do a close reading of Louise Rosenblatt and of some of the New Critics so that he can figure out how they differ. He seems to be quite very, very confused about fairly elementary matters in hermeneutics. Yikes. Perhaps he was speaking off the cuff before he had his coffee one morning.
LikeLike
cx: He seems to be quite confused about fairly elementary matters in hermeneutics. But perhaps, again, he was simply speaking thoughtlessly, off the cuff, and would dramatically edit his comments given the chance.
LikeLike
lol, well, that was my second guess, that he hadn’t finished his first cup of coffee yet; bad things happen when you speak pre-coffee.
LikeLike
Once again, great stuff Bob! I’ve been on your blog, and learned a lot there too. Have you written any books about education or considered doing so?
LikeLike
Here is the link to the scathing critique the NCTE did to the 2009 draft of the CC ELA standards. Did David Coleman and Susan Pimentel listen? http://web.archive.org/…/ReportCoreStdsRefs9%2019%2009.pdf
LikeLike
Here’s the link again if the one I just posted doesn’t work: http://web.archive.org/web/20131212205548/http://www.ncte.org/library/NCTEFiles/Resources/Standards/ReportCoreStdsRefs9%2019%2009.pdf
LikeLike
This link was working when I posted it earlier. Now it says it may have moved or been deleted. This is not good.
LikeLike
Don’t panic Sheila, it is still on the NCTE website (you don’t need the Wayback Machine):
Click to access Report_CoreStds_9-09.pdf
This is certainly an interesting document to look back upon. They understood many of the problems of Common Core from the beginning, yet it was impossible to unambiguously come out against it. Instead you get paragraphs like:
“Though it is not an exhaustive or even sufficient list of
skills, especially compared to the NCTE/IRA Standards for the English Language Arts or other NCTE positions, it is, for the most part, accurate as a list of a few things students should be able to do in college (though its alignment with research on workplace literacy is substantially more limited).”
I would think “The draft standards are an insufficient list of skills.” Full stop. Would have been sufficient and more appropriate to the moment.
Here was my take a couple of months later:
http://www.tuttlesvc.org/2009/09/10-reasons-why-you-should-care-about.html
LikeLike
Thanks for the link, Tom. I applaud you for realizing this back in 2009:
“We are inviting testing companies to determine the future of our schools with virtually no accountability or public input.
“These standards were developed by two testing companies, the College Board and ACT, with help from a nebulous non-profit, Achieve, Inc. It is essential to understand this when reading the Common Standards; it explains many of their odd choices. In the example above, the obvious interpretation is that they chose to define the standard as “support or challenge assertions” rather than “construct a response or interpretation,” as every international example they cited did, because the former is much easier and cheaper to score reliably on a standardized test.
“No high performing educational system in the world would consider giving testing companies this much control over their standards and curriculum. It is absurd.”
Unfortunately, Pear$on is gleefully proselytizing across the globe.
I’m a retired teacher from the RI School for the Deaf, and belong to the Coalition to Defend Public Education (Providence). Are you still in RI?
LikeLike
Debate teams base their ‘sides’ on intellectual persuasion, not accuracy. Perspective influences understanding. Therefore, on the premise that words are subject to interpretation, most fundamental questions can be argued (answered) differently. There are an infinite number of practical examples. Could the reform movement mean the end of creative argument in our schools and beyond?
LikeLike
“Tie up the loose ends,” it matters whether the note (you are reading) is from a macramé instructor or from a mob boss worried about possible informants! “
I love this example of the necessity of knowing about context when reading. Of course almost all of Bob Shepherd’s wonderful comments about reading also apply to art criticism and interpretation, except that students are usually more engaged in mustering words and generating phrases or sentences to describe what they perceive, feel, think, or imagine as they apprehend an example.
Shepherd’s wonderful post reflects his exceptionally deep knowledge of content, pedagogical moves, and assessment. On this blog, he is writing for teachers and other persons who are (in the main) well educated and probably thinking about the demands of the CCSS on mature students of literature and for students in middle to high school.
Students in earlier grades, and those from very different cultural/socioeconomic backgrounds in any grade, have acquired “conventional” knowledge and experiences… but these may not be relevant to the literary (artistic) selections in a test.
I am thinking of the wonderful writing of Richard Rodriquez whose accounts of his education illuminate these points better than I can here. Most of his referents for understanding school were not just mangled by his limited vocabulary in English, but by all of the differences between, home, neighborhood, and the lessons in schools, especially texts and tests.
This to say that the selection of items for tests, including snippets from literature or exemplars of the visual arts (reproductions), will usually end as Robert describes this process, and as Richard Rodriquez (serving as an illustration) might experience the items.
At best, some test items will be a dismal compromise about what should counts as evidence of learning.
At worst, some really outrageous stems/options/answers are allowed to stand for reasons of not noticing and cost in time and money fix them within a deadline.
And then, in-between are the bulk of test items requesting an “on demand” performance—based on assumptions about what the learner should have been taught in school, and presumably learned only in school, in a vertically aligned curriculum, meaning a system of grade-to-grade learning progressions each serving as a prerequisite for what follows (a key assumption in the CCSS).
Testing is a suitable way for the people who require and sell tests to hide their ignorance of learning, or indifference to what it entails, behind numbers and money.
Sometimes tests yield interesting but unsurprising results that are also insensitive to formal instruction in school (e.g., 90% of 9 year-olds taking a NAEP test in art who were shown a black and white photo of the iconic work, Mona Lisa, selected that name as the “correct title” from other options in the exercise booklet).
There is no doubt that testing is out of hand in the US. For 2015, PARCC is telling the nine states in its consortium to plan for 9 hrs 45 minutes for tests of the Common Core in grade three, escalating to 11hrs.15 minutes for grades 9-11. The Smarter Balanced consortium tests in 17 states are still expected to require about 8 hrs 30 minutes.
Nobody counts or cares much about the real influence of “on-demand, timed, and proctored tests” on students’ demonstrations of learning. These less-than-optimal conditions for thoughtful responses are not made better by the need to train students how to register an intended response with a mindless computer with a screen that displays now-you-see-it-now-you-can’t-see-it graphics.
PARCC has called for “performance” tests at about the 75% mark in the school year (technically allowing for 27 weeks of pre-test instruction) with end-of-course tests at the 90% mark (more than 3 weeks before the end of the school year).
Some additional blocks of time (not yet in place) will be needed to orient teachers and students to the computer interface with “sample” items, and to learn what to do if a student vomits on the keyboard, or what to do if the whole system crashes, and how to rehearse testing with the permitted accommodations for eligible students.
LikeLike
Read what I wrote to Bob, below.
LikeLike
Laura, one consequence to consider is the sale of textbooks. Districts will prefer materials created by the testing companies which will include the appropriate background necessary to complete the yearly assessments.
LikeLike
I just read this, and have little time to say what I wish to say. This kind of thinking, this type of conversations why this “teacher’s room is unlike any other. Where else does a teacher such as Bob Shepherd explain with such clarity and erudition the simple truth.
This question of SIGNIFICANCE as opposed to the author’s intentions or any literary analysis is an important one in understanding the curriculum that I wrote for the entire seventh grade at the East Side middle school where I taught Communication Arts/Literacy (English)from the day it opened. This curricula was studied by Harvard and the Pew researchers for the National Standards in the nineties, because they could not ignore the results of what I was doing… and that was enabling and facilitating the 13 year old kids to THINK, and then to find their best ideas, their personal ‘significance,’ not merely their own interpretations of the text or even their appreciation of the writing.
Bob, in order to ‘monitor’ their individualized reading, and create a reason for them to write which would be MEANINGFUL, something that offered ‘significance,’ I had them write a “Reader’s Letter,” to ME, each week, a la Nancy Atwell, (“In The Middle,” which I had read the months previous to setting up the CA curricula for the entire 6th and 7th grade the first year that the NYC magnet school opened.)
We had a small population at the start (which changed once word got out in Manhattan and the boroughs, about the excellence of the learning experience we provided.) Of course, it is GONE! Anything that worked evaporated!
https://vimeo.com/4199476
I will discuss more about the Readers Letters curricula which I designed, and scan some of letters, when I get my own site up and running, but for now, simply put, they wrote a letter each week, “Dear Mrs. Schwartz,” and I replied, to each and every student. I also wrote a “Dear Boys & Girls Letter,” each week modeling letter writing, and talking about things that were significant to me each week… asking them to reply if they wished to my ‘take-aways.’
‘Clear Expectations’ are the first principle of learning, I discovered when I became cohort.
I was clear and they liked the activity.
The criteria for these letters were clear…”talk about what you read, in clear sentences that show what the book meant to you.”
The significance of their individualized reading was important for ME TO KNOW, because there was no other way for me to KNOW if the read the book. THIS in the lingo is performance evaluation.
A plot summary would not do, but the character analysis that some of these bright kids described was really something to read… so I posted the letters outside by room — a 13 year old boy describes what he thinks of Daisy, and a 13 year old girl wonders why anyone reads Hemingway.
By June, each year, the letters outside my room offered the PROOF of the fallacy of tests as they evaluate critical thought, not just writing about reading. You see, IN addition to personal connections — (significance)criteria — I demanded a fuller letter, including sentences/paragraphs that demonstrated a connection to what was learned about writing as we read literature together.
How else was I to evaluate whether they appreciated the author’s craft, or were still just reading for fun… which is great… but not what MY evaluation tool needed.
Of course, the 30 letters, when collected showed the progression of their skills in spelling, vocabulary and grammar. Kids writing 50-100 words in September, were happily writing 1000 and more words by June… because they began to ‘tell’ me how they thought abut things, and i was more that a willing audience… it was what I wanted them to do… so they did…
I never gave a test!
Oh, there were a few quizzes to ensure that they memorized certain names, events and vocabulary, but the letters told me if they could use words to say what they needed to say, and IF they could use language with some finesse.
“My kids won every writing competition in NYC and beyond. So, in this day of confusion about WHAT IT TAKES TO TEACH WRITING, and HOW DO WE EVALUATE WRITING AND READING RESPONSES, I know the answer… BUT NO ONE ASKS ME, or you or THE TEACHERS… the professionals who have proven in their practice that they could do it….show the evidence that it works instead of some magic elixir like tests.
http://www.opednews.com/articles/Magic-Elixir-No-Evidence-by-Susan-Lee-Schwartz-130312-433.html
Ya know… I had a book offer: “Explain how did you do this”, Phillipa Stratton at Stenhouse told me,
It was simple… YOU explained it Bob, in this wonderful essay. These were kids — emergent minds — and they needed to find significance in order to discover their own ideas, to power their own stories. Motivation is key with kids, and a test score is NO MOTIVIATION. Kids liked explaining what was meaningful to them… hanging words on those kind of thoughts on paper.
They were writing stories, too,–as an end term activity. I am a playwright and love stories, so a big part of the creative writing curricula that I created, involved plotting stories, creating characters that powered the plot, , and using literary language and devices . But, as we read great authors like O’Henry, De Maupassant and Ray Bradbury and examined the author’s craft, these young writers began to analyze the books they were reading at home, as an audience for the author’s work!
YOU SHOULD READ THEIR LITERARY Analysis! Their letters stopped visitors in their tracks” This author took too long to introduce the major dramatic question,” one kid wrote… I gave up.”
“How old are these kids?” was a common response. Hee, hee… only 13 years old and knows a slow opening.
This is what is missing, what happens in a real classroom, when the right ‘essential question’ powers the ‘right’ activity — one where kids have clear expectations — and the RIGHT TOOL, which is not a test or a test-prep activity.
The best tool, used by a teacher-practitioner who stimulates classroom CONVERSATION AND COMMUNICATION (thinking) first, and then asks them to REFLECT again before they write, enables the kind of performance that is easily assessed.
Your essay got it right, and this is how it looks in a classroom.
LikeLike
Equally worrisome are the new Language Arts textbooks (with an interactive online component) that are mostly devoid of literary content. Last spring, all the old literature textbooks at my high school were removed from our classrooms and piled in enormous heaps in the hallways, to be later whisked away to “Surplus.” The new textbooks read like a social studies companion book, with real-life stories about people who have survived difficult times (mostly war-torn environments around the world). They all share character traits such as Fortitude. No classic short stories are included! There are only token excerpts of classic plays at the end of these textbooks, in modern translations. Our new writing booklet is nothing more than how-to templates specific to the three styles of writing to be tested in the spring: explanatory essays, argumentative essays, and literary analysis. No narrative essays will be tested, so we were told that we needn’t bother with them at all! We are instructed to follow explicitly the template styles, with students citing evidence directly from the articles that accompany each template-driven type of writing. We are also required to constantly read, grade, and then upload essay scores on data websites so that we and our students can be evaluated for “growth.”
Very few of us had the foresight to keep the old textbooks so that we may unobtrusively keep inspiring students to think and respond beyond the box with these wonderfully rich stories and poems. To make sure we are all “on track” we are given common planning times instead of individual planning times and are monitored by having to upload our daily lesson plans which must match the actual lessons that administrators “pop” into our classrooms to observe.
The lifeblood of the creative among us is being squeezed out drop by drop.
LikeLike
Scarey when one grasps that Koch and company are in control and will do everything they can to steer the minds of our students to what they value, their definitions of everything, democracy, what our founders envisioned and thought about human rights, science and … well, everything… ORWELL IS TURNING OVER IN HIS GRAVE.
They already control the media…what our citizens are told is happening, and what they should believe.
LikeLike
I have every single text that I used since 1963 in order to ‘teach’ writing, by exploring the author’s craft as our most expressive writers told of the human experience. I subbed in New York ‘s East Ramapo School District, and had all the texts of r every grade, so I could plan lessons ( required when this was a fine district which expected substitute teachers to ‘teach’ and continue the curricula.)
I have every lesson plan that I created as the seventh grade’s only English teacher (of literacy –Communication Arts) at East Side Middle School, a magnet school that opened in 1990. That curricula was studied by Harvard for Pew’s National Standard’s Research, so I have their analysis of how I met the standards for Learning as the NYC cohort in the national research.
I have copies of all the letters the students wrote offering their understanding of the ‘significance’ of the writing, as well as their appreciation of the authors’s craft.
I have videos of my practice, as does the LRDC at the UNiv of Pittsburgh, which filmed and observed my unique practice.
What I do not have is a job, as I was charged with incompetence and sent packing.
AND that is THE tale that underscores the end of public education.
if it works… it is destroyed.
My author’s page and my resume is at Oped News, where I write about education and link to this blog and all the important news. If you wish to contact me, you can message me at that page.
http://www.opednews.com/author/author40790.html
LikeLike
Once again, comics come to the rescue. (For anyone who thinks comics are low-brow, & not worth reading–remember all the former portrayals, where students would be “caught” reading comics behind their textbooks?–reading Archies & Charley Browns were how I learned to read at the age of 3! {Words such as “disgruntled” & “ubiquitous” frequently found in Archie Comics!}) There’s a syndicated strip called “One Big Happy” where a very bright little girl, Ruthie, gives multiple right (& clever!) answers to assigned homework (she has elongated–sometimes argumentative–conversations with both the “Homework Help” hotline, as well as the “Library Lady). “Grand Avenue” is another good example of out-of-the-box thinking–by both students and teacher. Oh, & let’s not leave out “Frazz,” in which one encounters not only sharp students & teachers, but also a truly wise custodian (who had once been a student at the school). BTW–take THAT Aramark & Sodexo–I’ve consistently had the honor of working with such sage maintenance personnel. CPS, bring our custodians back!!!
On tests–such as in life–there is ALWAYS more than one correct answer. Therefore, when assessments (of what-?)/Common Core & the whole rotten-to-the-core assign a “right” answer, the purpose of getting children “college and career ready” is nothing but an egregious fallacy–real life is nothing like a Pear$on test.
LikeLike
I think this one point needs to be repeated:
Real life is nothing like a Pear$on test.
LikeLike
A Pear$on test a day keeps reality away???? (but not Rheeality)
(Yeah, I know that was a bad one, couldn’t resist though)
LikeLike
This is the biggest problem of all with the tests, Ellen. What kids are asked to do on these tests bears almost no relation to what people actually do when they actually read and write well, so, ipso facto, the tests cannot be a valid measure of actual reading and writing ability.
LikeLike
And the NCTE, if it were worthy of the name, would long ago have said precisely that in a blistering denunciation of the entire enterprise.
LikeLike
rbmtk,
Surely you remember the Fabulous Furry Freak brothers comics?!?!
LikeLike
Glad to see that I’m not the only one who learned to read via comic books.
LikeLike
As Bob would say, using multiple choice items to quantify student proficiency of abstract and subjective ELA standards is like trying to measure Bill Gates ego with a triple beam balance.
LikeLike
It is hard for me to understand why we should try to get students to look at literature in a way that was popular in the 1950sand 1960s. Popular with literary critics, that is. Our students are not living in the mid 1900s and they are not going to become literary critics or even, for that matter, English teachers.
LikeLike
LOL. I call the CCSS literature standards “New Criticism for Dummies.” The approach had its uses, but it was deeply flawed, even when practiced well. When practiced badly–well, it makes a mockery of reading.
LikeLike
Coleman and his buddies were in way over their heads. Why don’t we just admit that and move on?
LikeLike
Thank you Bob Sheppard and Susan Lee Schwartz for your wisdom in teaching literature.
It is worth to repeat both of your wonderful expressions, such as:
From Bob Sheppard:
“This notion, that VIRTUE might be simply THERE to be discovered by an inquiring mind–an ABSOLUTE, unchanging TRUTH about which there could be no doubt on the part of one who has discovered it”, and
“TRUTH was ABSOLUTE, and preexisting, and there to be discovered by those with sufficient WIT and other QUALITIES to do so”
From Susan Lee Schwartz:
“This question of SIGNIFICANCE as opposed to the author’s INTENTIONS or any literary analysis is an IMPORTANT one in …COMMUNICATION”, and
“… and that was enabling and facilitating the 13 year old kids to THINK, and then to find their best ideas, their personal ‘SIGNIFICANCE,’ not merely their own INTERPRETATIONS of the text or even their APPRECIATION of the writing.”
In this particular thread, Bob and Susan are outstanding in their crystal clear explanation of the importance of SIGNIFICANCE, not INTENTION in Communication in
Literature and Arts.
In conclusion, “learners” need to have sufficient WIT and other QUALITIES which are developed with the help of ” EXPERIENCE/ WISE Teachers” in order to discover the ABSOLUTE TRUTH through practicing an inquiring mind/thinking, so that learners can distinguish their PERSONAL SIGNIFICANCE (=their own conviction), as opposed to people’s construing (=interpretations) in daily communication whether it is in literature or in real life. Back2basic
LikeLike
Robert – it is always illuminating to read your well thought out comments.
And your sense of humor shines through, making your points come to life.
Thank you for sharing.
LikeLike
kind of you, Ellen
LikeLike
There seems to be some confusion about the required use of “plausible distractors” when constructing MC test items. Only one of the three sets of response options below uses plausible distractors correctly. On set are a good example of what we are seeing in the Pearson ELA assessmensts.
________________________________________________
Shown below is a sentence using the word “flabbergasted”. Read the sentence carefully and use all the information in the sentence to help answer test question number one (1).
Sean was flabbergasted to learn that he was ranked third in his graduating class.
1) Which word best describes the meaning of the word “flabbergasted” as it appears in
the sentence provided?
a) surprised
b) astonished
c) stunned
d) amazed
e) disappointed
f) relieved
g) delighted
h) amazed
i) pushed
j) pulled
k) twisted
l) amazed
LikeLike
I know, I know, says the student wildly waving hand!
m. all of the above
(sorry folks, I’m in a corny mood right now)
LikeLike
What a beautiful illustration of one of the many serious problems with these tests. English teachers, who read well enough to see this problems need to say, in one voice: enough.
LikeLike
Thanks Bob. The last set of response options ( i – l) shows why distractors need to plausible. In my early days, before I was trained as an item writer, I would commonly use silly or stupid, or cutesy, “implausible” terms in if I couldn’t think of a good distractor. That 3rd distractor is often difficult to produce. Not an uncommon practice with teacher created tests.
The first set (a – d) is closer to what many of the Pearson items were like on their ELA test. When seasoned, veteran ELA teachers cannot agree on the correct response on multiple items, something is deeply wrong with the test. Ideally the terms “best” or “most likely” should not even be used in an item stem of an objective MC test. Pearson’s exams are littered with these – setting off alarm bells. If I submitted items like these to Measure Progress they wouldn’t have made it past the circular file. makes one wonder if this was just a hack job done on the fly or possibly intentional- in order to help produce the failure rate they wanted?
As you have stated before, using an MC test to measure the proficiency of subjective and abstract standards is simply the wrong tool for the job. Yet teachers careers and reputations now hinge on these poorly constructed tests. NYS should get its money back from Pearson, all test scores should be invalidated, and AIS assignments revoked. This should be a MAJOR SCANDAL.
I can’t begin to imagine the blowback once the PARCC and SBAC tests hit the fan this spring.Teachers and parents need to be relentless advocates regarding test transparency. False claims of test security will be used to avoid test scrutiny. If academic “rigor” is so important to this reform movement, you’d think that they would apply some to their test development.
LikeLike
Here’s what I think should happen: some Congressperson should subpena these testing companies and consortia, demanding copies of the ELA tests and of any reliability and validity “studies” performed on those, and then the tests should be subjected to item-by-item analysis and the studies should be subjected to competent peer review. Of course, the resulting reports would reveal that the tests in ELA are a complete scam being perpetrated on the taxpayers. The tests do not measure what they purport to measure and cannot as they are currently conceived.
LikeLike
yikes. “these problems” Oh, for a correction feature on WordPress!
LikeLike
NY, true enough, but one wonders why anyone would ever ask such a question. If the intent is to judge the student’s ability to guess the meaning from the context, then the context, here, is, of course, not sufficient to distinguish between answers like disappointed and relieved, stunned and amazed, and if that’s not the intent–if the question simply tests prior knowledge of the meaning of the word flabbergasted, then why bother with the context at all.
But there’s a deeper problem. Typically, for these tests, no one has done any item-by-item validation to provide any sort of evidence that individual items validly assess the particular standard that they are supposed to assess, and that’s yet another reason why the whole undertaking is a scam–a very lucrative scam being perpetrated by the testing companies with the ignorant assistance of politicians and educrats who ought to know better or, at least, ought to know that they don’t know enough to insist upon this crap.
LikeLike
And the SCAM is being perpetrated on a scale that is unprecedented, and at a cost of 325 million dollars paid to Pearson for the just the 2015, first go-round. I wonder what will happen when this money (a third of a billion dollars) runs out? Or is the Obama administration committing this kind of money on an annual basis?
LikeLike
Not to mention the billions and billions of dollars being spent to upgrade technology in order to administer them online. NYS has a 2 billion dollar bond issue on the ballot just for this purpose as we have yet to implement the actual PARCC tests. The true train wreck will hit this spring as tens of millions of youngsters sit down to computer terminals and iPads only to find that the computer screen format is ill suited to questions that require constant scrolling in ELA and math responses that involve the inputting of math symbols and equations. I’m still trying to picture nearly every eight year old in the nation keyboarding essay responses under timed conditions. The PARCC and SBAC will be particularly troublesome for ELLs and IEP students. It is a format almost designed to punish dyslexic students.
LikeLike
One of the ways in which professionals who specialize in hermeneutics go wrong, I think, is in their insistence upon a single approach to be championed above all others.There are many who would dictate to all what “meaning” is to mean. A wiser approach, I think, is an empirical one. One asks, “When people experience literary works, what kinds of things do they experience that they call “meaning”? And there will be many answers, many types of meaning. One really important answer is that people think, with good reason, that they recognize an authorial hand at work and recognize his or her intent. Only one who is willfully blind to the actual experiences that people have with texts could claim that the author is dead. But there are other answers as well. Meaning as significance–emotionally, spiritually, politically, philosophically, for example, is something entirely different.
Yet the two are connected. An author marshals materials with some intent, and that intent is typically to create an experience for a reader with a certain significance. Is there a “one true reading?” Well, that’s a useful notion–that there is a discoverable meaning that is the author’s intention–for such a notion preserves the making of literature as an act of communication, of transmission across the ontological gap between subjectivities. But the notion has its limits. You and I both enter into that world of the work, but we have different experiences that have differing significances to us. The relation between intent and significance is a complicated one. The ancient mariner is compelled to tell his story because there is a particular story that he wishes to tell, with particular significance to him. It’s a blasphemy against the human enterprise–against cultural transmission of that which has value–against all value–to dismiss the potential of communication of an intent. But a work escapes its author and its author’s proxies, the speakers in the work, and takes on varying significance, across time, in the lives of varying auditors. A great work lives inside its auditor and takes on its own life. Sometimes, it ramifies and deepens over time. Robert Frost wrote that he carried around Emerson’s “Brahma” in his head for decades before he understood it. At other times, the thing that once seemed so profound is dismissed, by the more mature person who has carried it around for some time, as too facile, too pat, too conventional, irrelevant, etc. And just as at different times in our own lives, works will differ in significance for us, so, too, they will take on differing significance in differing cultures–times and places. A work long dead can suddenly speak with a powerful voice to a particular age.
And, of course, authors can play all kinds of games with our notions about meaning. Rules are made to be broken. That fact should give theoreticians (and test makers) pause.
Reading well is important. Really important. Too important to be mucked with by test makers.
LikeLike
Lord, forgive them, the test makers, for they know not–literally, they know not–what they do.
LikeLike
Robert –
I am a member of two book clubs. Both are a disappointment. Although I love the diversity of our monthly book choices, the discussions are lacking. Basically the members simply indicate whether or not they enjoyed the selection with a brief discussion about the book. I want to talk about character development, style, plot twists, descriptive language, etc. I come with quotes I find particularly compelling. The rest of the group would prefer to eat dessert.
Now these are intelligent adults who enjoy reading. Let’s extrapolate to students who are forced to read a certain text. Their responses are similar to those of my adult friends. A few, potential English majors such as myself, might find enjoyment from the written word and the way a good author plies their trade. The rest are simply hoping for a compelling story which holds their interest. Anything else is considered a drudgery.
Perhaps a dish of ice cream would enhance their enjoyment.
LikeLike
Bob
Aside from eliminating CC and all related assessments (PARCC and SBAC), how would you fix this ELA testing mess? Or is it irreparable?
LikeLike
One should use the right tool for the job. Standardized tests are appropriate tools for judging elementary decoding ability. Beyond that–well, teacher grades are far better predictors of college success than are the best vetted of the standardized tests. If it were up to me, I would put the authority back in the hands of teachers and eliminate the standardized tests. It would be possible to put together some sort of portfolio system, but such a system should not strive for standardized products. Schools should not strive to produce standardized products. A complex, diverse society like ours needs diverse products from our schools, needs schools that build upon the strengths and interests of kids. Beyond the most basic of levels, if we treat kids as standardized products, we get mediocrity and absurdity, not individual initiative on the part of intrinsically motivated learners.
LikeLike
So, under NCLB/CC/PARCC/SBAC we are stuck with very bad and inappropriate tests that will drive very bad and inappropriate instruction. Until it dies the death of a thousand cuts.
LikeLike
And, of course, the whole notion of formal standards independent of content is absurd. For this reason alone, the whole approach was wrong from the start.
LikeLike
The ELA standards, like the state standards that preceded them, are almost entirely lists of highly abstractly and vaguely formulated “skills.” Wrong from the start. Such skills (and there are many that are nowhere addressed in these “standards”) are acquired incidentally when students are engaged in focused, guided ways, with substantive matters. The whole approach is like trying to teach people to ride bicycles by having them think in the abstract about bicycle riding skills and having them learn the names and functions of the muscles and nerves and bones and other body parts involved. Fundamental rethinking needed to be done that was not done. So, as a result, de facto, Achieve et al., have made this list of abstractly formulated skills into the curriculum. Snippets of literature and “informational text” are treated at random, out of context, as interchangeable and unimportant occasions for applying skills. The whole of approach is absurd.
LikeLike