Robert Shepard, author and curriculum designer, has
prepared an essay exam for the corporate reformers who think they
know how to redesign American education.
He writes:
As a member of the Billionaire Boys’ Club, or as one of the paid associates
of the BBC, you . . .
1. believe that that
extraordinarily complex skills like reading and writing ability can
be validly and reliably measured by simple, objective
tests.
Explain how that could possibly be so.
Please draw upon your extensive knowledge of the relevant
scientific literature.
2. believe that innovation comes about when free persons conceive of varied goods
and services that compete with one another in a free market in
which users choose the goods and services that they wish to
purchase and use.
Explain how this belief can
be reconciled with a) a single set of mandatory national standards
for all students, b) a single set of mandatory high-stakes national
tests, c) a single national database of all student test scores and
responses, and d) scripted literacy lessons that all teachers must
follow to the letter.
3. believe that all
students should follow the same standards and take the same
tests.
Explain how this belief can be
reconciled with the fact that students differ enormously in their
backgrounds, in their developmental levels, in their gifts and
interests and propensities, and in the goals that they and their
parents have for their futures.
4. believe that national standards do not narrow and distort curricula and
pedagogy.
Please answer the following questions:
If standards do not drive (and so narrow and distort)
curricula and pedagogy, why create them?
If they do drive curricula
and pedagogy, how can a single set of predetermined standards be
better than ANY alternative set that might be developed by ANY
OTHER expert or group of experts in education and particular
subject matter?
5. believe that our schools
are failing.
Explain how can this belief can
be reconciled with the fact that, when results on internationally
norm-referenced exams in reading, mathematics, and science are
corrected for the socio-economic levels of students taking the
exams, U.S. students consistently score at the top or very near the
top?
6. believe that a small group of persons
appointed by a committee of politicians should be empowered to
create standards that overrule and render irrelevant the judgments
about desirable outcomes in particular courses of study made by
professional teachers, curriculum developers, and curriculum
coordinators.”
Why?
Diane Ravitch's blog 
Love, love, love it!
These could all be honestly answered in three words: “I love money”.
Yes, indeed. They actually don’t “believe” in any of the offered statements. They simply believe there is money top be made and they will lie, cheat and steal from our children to get it.
Sorry, that would be “money to be made”, not, ” money top be made”.
Some of them, yes. But there are others who seem actually to believe that one set of invariant standards and tests for all is sound educational policy. That anyone would believe something so patently absurd strains credulity, but I have met people who suffer from this delusion.
Robert,
How much do I get paid to “score” the answers?
Duane
P.S. I hope that you are charging the BB’s at least $10G per question as my scoring techniques are quite expert.
We’ll be outsourcing the scoring to some great folks in India. We have rubrics to ensure that only answers that follow rigid formulas will receive passing scores.
Applying a standardized grading rubric to these essays would be hilarious. Ironically, it would likely provide the answerers with an amazing loophole. They could answer using topic sentences, three examples per paragraph and a conclusion that restates everything they just wrote and still completely avoid the question…not say anything at all…and get a passing or “good” grade.
So, I’d say it needs it needs real graders. 🙂
All this drives a person to wonder if this is a big conspiracy to dumb everyone down to control them better. One would think that the whole point of education – and the whole point of “reading for information”- is so that the student can think critically about the information the student found. Everywhere I look, that step is missing.
When English teachers start to realize that it is still within their power to use Common Core to teach kids how to think critically about the society that has been set up by and for the 1%, then there will be a Great Backpedaling on “reading for information”.
Well said, Emmy!
One of my recurring fantasies, Emmy, is this: Rumi or William Blake or Spinoza or Wittgenstein or Derrida or Foucault or Einstein or Feynman answers the boilerplate high-stakes essay exam question. Anyone of any genius would, of course, fail to meet the criteria of the rubric and evince a determination not to regurgitate the expected standardized answer.
Kids differ. They differ a lot. And to the extent that we are doing our jobs as teachers, we help them to develop and accentuate those differences. Our schools should be custom shops, not mills for creating identical machine parts.
Emmy,
I most definitely think there lies within a conspiracy! I’ve said it for years now. After all, how are the 1% going to stay the 1% if they don’t have control of the “general” population! And how do you keep control?–when the “general” population reaches a point where they have virtually no critical thinking skills to discern important information regarding job analysis, benefits analysis, political analysis, etc.!
Thank you, Robert, for such an excellent post!!!
Duane, if you are a warm body and are willing to work for minimum wage, then you can score the exams. We have a fifteen-minute training course that we’ll put you through. And $10G, you have to be kidding. We charge billions for our expert exam-creation services and require multi-year contracts. The creation of tests is an expensive undertaking. You have no idea how many politicians one has to buy in order to get a new exam of the ground!
off the ground
Robert – The fact that a skill is highly complex does not inply that capability at such skill cannot be objectively and easily determined. Human speech is the most complex muscular activity that we know about in all of biology. Nevertheless native speakers of all languages can easily detect even slight variations from the normal pattern of their languages. Even very minute differences in the timing of transitions between the initial and final sounds in the same diphthongs between say native speakers of English and say someone whose first language is French can be easily detected by almost all native speakers of English.
Both standing and walking not to mention running are fantasically complex muscular activities. Nobody really has a clue as to how human beings who have a basically quadruped body are able to stand up without immediately falling om their faces. Nevertheless few of us have any problems determing whether someone can stand up.
Running antelopes are engaged in a fantastically complicated activity but a coyote who is chasing them can very quickly determine which of the antelope is the least competent runner and concentrate on that antelope.
It is an obvious fallacy that the complexity of an activity implies that determing how well it is performed is not possible.
It may be the complexity of the desired outcome that causes the problem with measurement. Or do you really believe that the purpose of public education is to get young people to score well on bubble tests?
I’m not necessarily endorsing Robert D. Shepherd’s line of reasoning in this instance, but your standing-up and running analogies don’t hold up very well either. Skill in writing an essay about literature is a complex set of tasks that can’t be evaluated properly just by eyeballing the result. To that extent, it isn’t analogous to walking, standing up, or even speaking a language.
At the same time, if it’s true that it isn’t all that hard to roughly evaluate a student’s writing and interpretive skills–say, with a simple in-class writing assignment that can be graded locally by teachers–then we are in the process of wasting billions of dollars and millions of hours of human effort on unnecessary testing and overwrought instructional design.
And why is that? Because it pays. It’s a classic racket–a person or entity with a source of income is coerced into paying for otherwise unneeded services under the threat of harm. Consultants, publishers, testing and tech companies, charter operators, and more–they’re all cashing in on the punitive “reforms” that have been written into law.
Lobbyists, “non-profit” foundations, and self-interested political donors are bound and determined to keep the rackets going (some with the long range goal of lowering taxes, some with the immediate goal of capturing markets and extracting money from the public that pays those taxes, some both). All this to the detriment of children, who are being unduly stressed and falsely labeled as failures (see New York state) to make the “reforms” seem necessary. This sad story is well documented in Diane’s new book.
The people who love these standards and these tests don’t know enough about reading, writing, speaking, listening, and thinking to understand what poor instruments these are, how deeply flawed, how full of glaring lacunae they are, how much they fall short of beginning to measure the range of ability that exists within each of these areas of human accomplishment.
it’s a classic racket. Well said, Randal.
Jim, consider any part of the ability to read–for example, syntactic fluency. Can the student comprehend sentences that contain relative clauses? correlative constructions? appositives? coordinated main ideas? infinitive clauses? short clauses? nominative absolutes? Nothing in the instruments being used even begins to capture the relevant information. It’s as though we were measuring ability to play baseball based on ability to catch a popup fly.
“It’s as though we were measuring ability to play baseball based on ability to catch a popup fly.”
Noel Wilson uses sports examples to show the how absurd “measuring” the teaching and learning process using one small part of the assessment process, i.e., multiple guess bubble-in standardized tests.
And that is why I implore all to read the complete study (as I am completely rereading it now) and not just my meager summary.
And Jim, you missed my point entirely. As I said, kids differ. They differ a lot, and the point of schooling should be to recognize, celebrate, and build upon these differences. There are innumerable different ways to be a good reader, writer, speaker, listener, and thinker. Our goal should NOT be to produced standardized students. We’re not running a CNC machine for the production of screws in conformity with thread standards. We’re running schools–schools that are meant to produce citizens of an extraordinarily complex, diverse, pluralistic society.
Actually the determination of at least the general level of a person’s reading and writing ability doesn’t seem particularly difficult. Most of us can probably make a fairly good guess as to an individual’s reading and writing ability just based on a fairly short and purely oral conversation.
Really, Jim? Well, who reads better, E.D. Hirsch, Jr., or Harold Bloom. They disagree entirely about what Blake’s Songs of Experience are saying. And who reads better, little Yolanda, who tears through the Lemony Snicket books and understands a great deal about characterization or little Kwame who never reads fiction, can’t understand what all the fuss is about with regard to fiction, but can follow the directions in the Lego set precisely? Different skills. Vastly different abilities.
Robert – The fact that an extreme outlier like Feynmann might possibly not do well on a standardized test is a silly point of very little relevance given the very low frequency of such individuals.
Jim, ALL people vary. They vary enormously across an enormous range of abilities. There is not one general intelligence factor. There are not seven of these. There are, literally, thousands and thousands of intelligences. See Minky’s Society of Mind for a basic primer.
Robert,
We do group almost all students together based on age in a single class and teach them all in one classroom. If we take this enormous variability seriously, wouldn’t we have to change the practice of age based grade standards of any sort or do you think there is enough similarity between students when they are younger that we can assign them to a single classroom with a single lesson plan?
TE. that’s a GREAT question. We really need to be serious about thinking up alternatives to these invariant systems. I was thinking, recently, about those outlier kids who are born with exceptional mathematical abilities. We know that they exist, and we know that such people–the little Eulers and Ramanujans–can make enormous contributions if their talents are recognized and built upon from an early age. I know that we have gifted programs, but such kids are REALLY exceptional. Do you know whether there are any programs for kids like this who are extreme outliers?
Programs are likely to be local. Rich, dense public school districts are likely to have strong programs, smaller less wealthy probably little or nothing. My district has nothing, though students do take classes at the local university while in high school.
Nationally the only thing I know about is Bard College at Simon’s Rock. The link is here:http://simons-rock.edu.
TE, I would like to see that grade system replaced with one in which kids ALL have IEPs and, throughout their schooling, from at least middle school on, have a team of academic counselors who meet regularly to advise the child and his or her parents upon a course of study. I would like to see matriculation from particular grades and levels of schooling, as well as letter grades, replaced by competency certificates to in subfields of disciplines to be earned through a combination of coursework and juried portfolio work. I haven’t thought this through entirely, but those are the lines that I have been thinking along.
Robert @ 4:34. That would pretty much be my plan if I were anointed ruler-of-all-things-educational! Instead, our country has lowered the bar tremendously to “what is the least expensive solution we can provide?” So yes, to the commenters above, it is possible to distinguish a basic level student from an advanced student on a standardized test. However, as many practicing teachers say here often, they can do that themselves just by observing their students. So, how much value are we extracting from all of this money we are spending on testing? Bill Gates should fund me to do a study that compares standardized test scores to teachers’ mid-year ranking of students’ ability. If the correlation is high then we ought to ask ourselves why the hell we are spending all of this money and effort on testing if we can get the same information cheaper, faster and with less stress on the students. Somehow I know my check got lost in the mail…
So neither you or Robert would like to see public education go back to where it was, say, 15 years ago?
Continuous improvement, TE, from the bottom up, not from the top down.
There have been a number of top down changes involving access to education that most who post here likely aprove. How would you decide which decisions local school districts should make for themselves and which should be imposed by federal legislation and federal courts?
“They vary enormously across an enormous range of abilities.”
Not only “across an enormous range of abilities” but also over time and place due to myriad factors outside the control of the individual that can effect anyone at anytime, accidents, illnesses, affective states, etc. . . .
What combinations of circumstances will make you less fluent in Spanish than your students in first year Spanish?
Jim, you seem to be assuming that I mentioned Feynman et al. as part of an argument against the validity of these tests. I did not. I mentioned them because bright people like Feynman understand better than most do why Einstein said, “I believe in standardizing automobiles, not people.” They are outliers that illustrate especially well a general principle–that we need diversity of outcomes from schooling, not conformity. The Common Core is a recipe for enforcing mediocrity on everyone.
The lack of validity of the tests rests on other arguments.
And all one needs to do to understand that total lack of validity is read and understand what Noel Wilson has to say in “Educational Standards and the Problem of Error” found at: http://epaa.asu.edu/ojs/article/view/577/700
Jim,
Low frequency??? Obviously, you must not be a teacher. Ask a teacher in a lower socio-economic school. The little Kwame’s are many!– NOT “very low frequency”!! Also, your viewpoint seems to say “very low frequency”–then disregard and discard!
For the considered judgment and unmatched wisdom of one of the leading education rheephormers of this most cage busting achievement gap crushing twenty first century, I turn to a giant in the field of Sacred EduMetrics:
[start quote] “A reading comprehension test is a reading comprehension test. And a math test in the fourth grade—there’s not many ways you can foul up a test… It’s pretty easy to ‘norm’ the results.” [end quote]
President of the United States of America, George W. Bush Jr., in March 2001.
[Cited in Daniel Koretz, MEASURING UP: WHAT EDUCATION TESTING REALLY TELLS US (paperback, 2009, p. 7).]
Where do you think State Commissioner John King got the idea that Common Core = Montessori?
🙂
Randal Hendee – I agree with you that extensive standardized testing is a waste of time and money. But this isn’t because the determination of a person’s reading and writing ability is paticularly difficult or problematical. I am not a supporter of extensive standardized testing but it’s simply nonsense to say that abilities such as reading/writing ability or for that matter mathematical ability are something so etheral that they defy empirical measurement.
I did not say that they defy empirical measurement, Jim. I said that these tests are not valid measures of them. Different statement, but go ahead and continue setting fire to that straw man if you wish to. Enjoy.
Jim,
Robert’s right, he hasn’t rejected (unfortunately) the idea that we can “measure” these things (maybe some day).
I’m the one (in channeling Wilson) who says that it is logically impossible to “measure” these sorts of things.
Duane
In general the ability level of different individuals in regard to both reading\writing ability and ability to solve mathematical problems is not at all difficult to ascertain.
It is my contention, Jim, that reading and writing and speaking and listening and thinking are all very complex and that a general label–good reader, bad reader–doesn’t capture much of anything of any relevance, that we need, instead of these crude summative tests, diagnostics and formative tests, embedded in the curriculum, that are much, much more granular. Is a person who is an adult and can read an operating manual for a conveyor belt but who has no understanding of the symbol system that relates the human life cycle to the cycle of the seasons (and to whom a vast amount of the world’s literature is, ipso facto, inaccessible) a good reader? Well, yes. But this person is a different sort of reader. Our tests will tell us whether something is a tree. They won’t tell us whether we can harvest syrup from it for human consumption.
Robert – The syntax of your “sentences” baffles me. Hiowever since standardized tests provide a numerical score they are certainly more granular than a dichotomous good reader/bad reader classification.
Jim, if you think that a score on one of these standardized tests tells people much of value with regard to their specific attainments and deficiencies, then you simply aren’t very familiar with the tests or with reading. My good/reader bad reader is short for a proficiency cut off. Kids above the line are proficient. Those below the line are not. Those single scores are the scores that matter. They are the ones used for making high-stakes decisions. And they are extraordinarily crude. They are equivalent to “good reader” or “bad reader,” and that was my point. Those scores are simply too crude to be of any use.
Sorry you find my “sentences” so baffling. If you are finding the reading challenging, there are some high-stakes reading tests that you can take, and these will place you into reading curricula from one of the big-box publishers with materials aligned to the Common Core.
That “numerical score” is a chimera, a ghost, a duende, fantasy, bogystar, bubblestar, delusionstar, fabricationstar, fancystar, figmentstar, hallucinationstar, illusionstar, miragestar, monsterstar, monstrositystar, rainbowstar, snarestar, specterstar, fata morganastar,
fool’s paradisestar (from thesaurus.com) and certainly is not a certainty, fact, reality or truth.
Robert – I tried to count the number of words in your first “sentence” and I lost count at something over 110. Perhaps diagraming that sentence should be made a question on a standardized test.
You make a good point, Jim! 🙂
Well, at times I have picked a page randomly out of a story or novel in Spanish. Usually the sentence I point to randomly out of that random page is over a half page long of #7-8 font. I have no trouble understanding what Robert writes.
Sorry Robert, I missed the period after “granular”. But Robert, you need an editor.
Jim, this is a blog. It’s conversation.
But let me go at that again. Consider two people, X and Y. X is an English professor. He knows a lot about how writers, throughout history, have related the human life cycle to the cycle of the seasons, and he knows a lot about early religion and thus about the deep roots of the symbol system that relates these too. Y is an industrial maintenance supervisor. Y can read with ease an operating manual for a complex piece of electrical, mechanical, or hydraulic machinery and follow its directions when doing troubleshooting or repair. To X, the operating manual on that hydraulic cylinder actuated pump might as well be the Voynich manuscript. To Y, the same would be true of, say, Dylan Thomas’s “Fern Hill.”
OK. Which is the better reader?
The point is that there are many, many ways in which to be a good reader. Arguably, those who are not familiar with the cycle of the seasons/life cycle set of tropes have missed an essential part of a literary education. If one does not know about this symbol system, then much of the orature and literature of the world will be a closed book. So, should knowledge of that symbol system be tested by a 12th-grade high stakes test of reading ability? Should the ability to produce a line in iambic pentameter be measured by the high-stakes test in writing? These are key skills, after all. And certainly, possessing them would enrich one’s life. But so would being able to understand directions for disassembling a hydraulic pump.
There are many, many ways in which to be a good reader. To understand what people’s abilities are, even at a very basic level, one needs a very wide range of granular diagnostics, not invariant, simple and simple-minded summative tests.
these two, not too
“There are many, many ways in which to be a good reader. To understand what people’s abilities are, even at a very basic level, one needs a very wide range of granular diagnostics, not invariant, simple and simple-minded summative tests.”
Pearson says: “I LIKE where this is going!”
Ron, I would like it a LOT if the testing companies got out of the high-stakes summative testing business and got into the business of creating valid, highly granular, specific diagnostics. Summative testing should be done by teachers based upon their course content and their knowledge of their students.
“Arguably, those who are not familiar with the cycle of the seasons/life cycle set of tropes have missed an essential part of a literary education. If one does not know about this symbol system, then much of the orature and literature of the world will be a closed book.”
If that’s arguable, I’ll argue against it. For me, this is a reductive and exclusionary approach to literature. Audiences don’t NEED a formal understanding of a particular symbol system or any other kind of literary machinery to benefit from a work of literature. Sure, English teachers should talk about the workings of literary art, but the student’s attempt to understand a work should take precedence over the teacher’s attempt to convey knowledge about it.
I realize your main point is in opposition to high-stakes testing on these topics, and that you’re hedging your assertions by using the terms “arguably” and “much of.” I just think there’s too much emphasis on formal analysis of how meaning manifests itself in literary works. For me, this is one of the big flaws of the “CCS,” especially as this practice is being pushed down to the early grades. It is both reductive and counterproductive, in that it gives a distorted view of what literature is, and it’s a good way to kill a child’s interest in it.
As for, “To understand what people’s abilities are, even at a very basic level, one needs a very wide range of granular diagnostics…” This would be another huge waste of money and energy. Better to set up reading clubs in the classroom than to subject the kids to a battery of reading tests.
Formal diagnoses aren’t generally needed unless a teacher notices a serious disability. Maybe you’re not in favor of implementing “a wide range of granular diagnostics,” though. I know I’m not.
Ron, I agree entirely. Perhaps I was not clear. I meant for people to take from my comments to the point that there are MANY such understandings, non of them currently tested, that make for MANY different sorts of adult literacy. God forbid that any of these be instantiated into an invariant summative test!
Diagnostics can provide teachers with news they can use. However, place even those in the hands of the fool educrats, and they will be putting them in kids’ permanent records, labeling kids based on them, treating them as high stakes summative assessments, and subjecting kids to hours and hours of them, So, yes, Ron, I have my doubts even about diagnostics, though they are MUCH to be preferred to summative assessments.
When they are together behind closed doors, they remind each other that those with the money have the power to do anything they want. We are now paying for Private schools with your tax dollars so they can pay less! They are changing voter laws for their benefit. They are taking away the voices of Educators by getting rid of educators right to due process and on and on.
YEP!!