Bob Shepherd, veteran curriculum and assessment designer, is now teaching in Florida. Here are his thoughts on Common Core and the testing obsession:
How to Prevent Another PARCC Mugging: A Public Service Announcement
The Common Core Curriculum Commissariat College and Career Ready Assessment Program (CCCCCCRAP) needs to be scrapped. Here are a few of the reasons why:
1.The CCSS ELA exams are invalid.
First, much of attainment in ELA consists in world knowledge (knowledge of what—the stuff of declarative memories of subject matter). The “standards” being tested cover almost no world knowledge and so the tests based on those standards miss much of what constitutes attainment in this subject. Imagine a test of biology that left out almost all world knowledge about biology and covered only biology “skills” like—I don’t know—slide-staining ability—and you’ll get what I mean here. This has been a problem with all of these summative standardized tests in ELA since their inception.
Second, much of attainment in ELA consists in procedural knowledge (knowledge of what—the stuff of procedural memories of subject matter). The “standards” being tested define skills so vaguely and so generally that they cannot be validly operationalized for testing purposes as written.
Third, nothing that students do on these exams EVEN REMOTELY resembles real reading and writing as it is actually done in the real world. The test consists largely of what I call New Criticism Lite, or New Criticism for Dummies—inane exercises on identification of examples of literary elements that for the most part skip over entirely what is being communicated in the piece of writing. In other words, these are tests of literature that for the most part skip over the literature, tests of the reading of informative texts that for the most part skip over the content of those texts. Since what is done on these tests does not resemble, even remotely, what actual readers and writers do in the real world when they actually read and write, the tests, ipso facto, cannot be valid tests of real reading and writing.
Fourth, standard standardized test development practice requires that the testing instrument be validated. Such validation requires that the test maker show that the test correlates strongly with other accepted measures of what is being tested, both generally and specifically (that is, with regard to specific materials and/or skills being tested). No such validation was done for these tests. NONE. And as they are written, based on the standards they are based upon, none COULD BE done. Where is the independent measure of proficiency in CCSS.Literacy.ELA.11-12.4b against which the items in PARCC that are supposed to measure that standard on this test have been validated? Answer: There is no such measure. None. And PARCC has not been validated against it, obviously LOL. So, the tests fail to meet a minimal standard for a high-stakes standardized assessment—that they have been independently validated.
2. The test formats are inappropriate.
First, the tests consist largely of objective-format items (multiple-choice and EBSR). These item types are most appropriate for testing very low-level skills (e.g., recall of factual detail). However, on these tests, such item formats are pressed into a kind of service for which they are, generally, not appropriate. They are used to test “higher-order thinking.” The test questions therefore tend to be tricky and convoluted. The test makers, these days, all insist on answer choices all being plausible. Well, what does plausible mean? Well, at a minimum, plausible means “reasonable.” So, the questions are supposed to deal with higher-order thinking, and the wrong answers are all supposed to be plausible, so the test questions end up being extraordinarily complex and confusing and tricky, all because the “experts” who designed these tests didn’t understand the most basic stuff about creating assessments–that objective question formats are generally not great for testing higher-order thinking, for example. For many of the sample released questions, there is, arguably, no answer among the answer choices that is correct or more than one answer that is correct, or the question simply is not, arguably, actually answerable as written.
Second, at the early grades, the tests end up being as much a test of keyboarding skills as of attainment in ELA. The online testing format is entirely inappropriate for most third graders.
3. The tests are diagnostically and instructionally useless.
Many kinds of assessment—diagnostic assessment, formative assessment, performative assessment, some classroom summative assessment—have instructional value. They can be used to inform instruction and/or are themselves instructive. The results of these tests are not broken down in any way that is of diagnostic or instructional use. Teachers and students cannot even see the tests to find out what students got wrong on them and why. So the tests are of no diagnostic or instructional value. None. None whatsoever.
4. The tests have enormous incurred costs and opportunity costs.
First, they steal away valuable instructional time.
Administrators at many schools now report that they spend as much as a third of the school year preparing students to take these tests. That time includes the actual time spent taking the tests, the time spent taking pretests and benchmark tests and other practice tests, the time spent on test prep materials, the time spent doing exercises and activities in textbooks and online materials that have been modeled on the test questions in order to prepare kids to answer questions of those kinds, and the time spent on reporting, data analysis, data chats, proctoring, and other test housekeeping.
Second, they have enormous cost in dollars. In 2010-11, the US spent 1.7 billion on state standardized testing alone. Under CCSS, this increases. The PARCC contract by itself is worth over a billion dollars to Pearson in the first three years, and you have to add the cost of SBAC and the other state tests (another billion and a half?), to that. No one, to my knowledge, has accurately estimated the cost of the computer upgrades that will be necessary for online testing of every child, but those costs probably run to 50 or 60 billion. This is money that could be spent on stuff that matters—on making sure that poor kids have eye exams and warm clothes and food in their bellies, on making sure that libraries are open and that schools have nurses on duty to keep kids from dying. How many dead kids is all this testing worth, given that it is, again, of no instructional value? IF THE ANSWER TO THAT IS NOT OBVIOUS TO YOU, YOU SHOULD NOT BE ALLOWED ANYWHERE NEAR A SCHOOL OR AN EDUCATIONAL POLICY-MAKING DESK.
5. The tests distort curricula and pedagogy.
The tests drive how and what people teach, and they drive much of what is created by curriculum developers. This is a vast subject, so I won’t go into it in this brief note. Suffice it to say that the distortions are grave. In U.S. curriculum development today, the tail is wagging the dog.
6. The tests are abusive and demotivating.
Our prime directive as educators is to nurture intrinsic motivation—to create independent, life-long learners. The tests create climates of anxiety and fear. Both science and common sense teach that extrinsic punishment and reward systems like this testing system are highly DEMOTIVATING for cognitive tasks. The summative standardized testing system is a really, really backward extrinsic punishment and reward approach to motivation. It reminds me of the line from the alphabet in the Puritan New England Primer, the first textbook published on these shores:
F
The idle Fool
Is whip’t in school.
7. The tests have shown no positive results.
We have had more than a decade, now, of standards-and-testing-based accountability under NCLB. We have seen only miniscule increases in outcomes, and those are well within the margin of error of the calculations. Simply from the Hawthorne Effect, we should have seen SOME improvement!!! And that suggests that the testing has actually DECREASED OUTCOMES, which is consistent with what we know about the demotivational effects of extrinsic punishment and reward systems. It’s the height of stupidity to look at a clearly failed approach and to say, “Gee, we should to a lot more of that.”
8. The tests will worsen the achievement and gender gaps.
Both the achievement and gender gaps in educational performance are largely due to motivational issues, and these tests and the curricula and pedagogical strategies tied to them are extremely demotivating. They create new expectations and new hurdles that will widen existing gaps, not close them. Ten percent fewer boys than girls, BTW, received a proficient score on the NY CCSS exams–this in a time when 60 percent of kids in college and 3/5ths of people in MA programs are female. The CCSS exams drive more regimentation and standardization of curricula, which will further turn off kids already turned off by school, causing more to turn out and drop out.
This message not brought to you by
PARCC: Spell that backward
notSmarter, imBalanced
AIRy nonsense
CTB McGraw-SkillDrill
MAP to nowhere
The New Scholastic Common Core Achievement Test (SCCAT)
The Bill and Melinda Gates Foundation (“All your base are belong to us”)

Wait till he finds out about Competency based Education and Mastery.
LikeLike
Didn’t you mean Cowardly Based Education and Misery???
LikeLike
Bob hits the bull’s eye once more. Great summation of the flaws in the boondoggle known as CCSS/PARCC/SBAC.
LikeLike
Agree-Love your New Criticism Lite, or New Criticism for Dummies
LikeLike
Thank you, Bob, for so clearly and authoritatively summing up the travesty of Common Core/PARCC/SBAC. I also concur with the concerns of howardat58–from what I’ve been reading, the powers-that-be are co-opting the concerns of parents and teachers about the PARCC/SBAC and over-testing (though not saying much about the Common Core) and are forging ahead with the reasonable sounding Competency/Proficiency Based agenda of so-called personalization, student-centered, anytime anywhere (digital) learning. This will truly be the death knell of actual critical thinking, curiosity, and creativity. The data collection menace from all of this online/digital learning is truly chilling. I wrote a piece on personalization and the new RI Strategic Plan for Public Education: 2015-2020 yesterday. (with gratitude to Emily Kennedy Talmage for her incredible research and blog posts) http://www.rifuture.org/what-is-competency-based-education.html
LikeLike
This is one of the most reasoned as to why CC should be scraped.
LikeLike
It is one of the most reasoned, no doubt, but THE MOST REASONED CASE against educational standards and standardized testing has been available since 1997 when Noel Wilson published his never refuted nor rebutted dissertation “Educational Standards and the Problem of Error” found at: http://epaa.asu.edu/ojs/article/view/577/700
Brief outline of Wilson’s “Educational Standards and the Problem of Error” and some comments of mine.
1. A description of a quality can only be partially quantified. Quantity is almost always a very small aspect of quality. It is illogical to judge/assess a whole category only by a part of the whole. The assessment is, by definition, lacking in the sense that “assessments are always of multidimensional qualities. To quantify them as unidimensional quantities (numbers or grades) is to perpetuate a fundamental logical error” (per Wilson). The teaching and learning process falls in the logical realm of aesthetics/qualities of human interactions. In attempting to quantify educational standards and standardized testing the descriptive information about said interactions is inadequate, insufficient and inferior to the point of invalidity and unacceptability.
2. A major epistemological mistake is that we attach, with great importance, the “score” of the student, not only onto the student but also, by extension, the teacher, school and district. Any description of a testing event is only a description of an interaction, that of the student and the testing device at a given time and place. The only correct logical thing that we can attempt to do is to describe that interaction (how accurately or not is a whole other story). That description cannot, by logical thought, be “assigned/attached” to the student as it cannot be a description of the student but the interaction. And this error is probably one of the most egregious “errors” that occur with standardized testing (and even the “grading” of students by a teacher).
3. Wilson identifies four “frames of reference” each with distinct assumptions (epistemological basis) about the assessment process from which the “assessor” views the interactions of the teaching and learning process: the Judge (think college professor who “knows” the students capabilities and grades them accordingly), the General Frame-think standardized testing that claims to have a “scientific” basis, the Specific Frame-think of learning by objective like computer based learning, getting a correct answer before moving on to the next screen, and the Responsive Frame-think of an apprenticeship in a trade or a medical residency program where the learner interacts with the “teacher” with constant feedback. Each category has its own sources of error and more error in the process is caused when the assessor confuses and conflates the categories.
4. Wilson elucidates the notion of “error”: “Error is predicated on a notion of perfection; to allocate error is to imply what is without error; to know error it is necessary to determine what is true. And what is true is determined by what we define as true, theoretically by the assumptions of our epistemology, practically by the events and non-events, the discourses and silences, the world of surfaces and their interactions and interpretations; in short, the practices that permeate the field. . . Error is the uncertainty dimension of the statement; error is the band within which chaos reigns, in which anything can happen. Error comprises all of those eventful circumstances which make the assessment statement less than perfectly precise, the measure less than perfectly accurate, the rank order less than perfectly stable, the standard and its measurement less than absolute, and the communication of its truth less than impeccable.”
In other words all the logical errors involved in the process render any conclusions invalid.
5. The test makers/psychometricians, through all sorts of mathematical machinations attempt to “prove” that these tests (based on standards) are valid-errorless or supposedly at least with minimal error [they aren’t]. Wilson turns the concept of validity on its head and focuses on just how invalid the machinations and the test and results are. He is an advocate for the test taker not the test maker. In doing so he identifies thirteen sources of “error”, any one of which renders the test making/giving/disseminating of results invalid. And a basic logical premise is that once something is shown to be invalid it is just that, invalid, and no amount of “fudging” by the psychometricians/test makers can alleviate that invalidity.
6. Having shown the invalidity, and therefore the unreliability, of the whole process Wilson concludes, rightly so, that any result/information gleaned from the process is “vain and illusory”. In other words start with an invalidity, end with an invalidity (except by sheer chance every once in a while, like a blind and anosmic squirrel who finds the occasional acorn, a result may be “true”) or to put in more mundane terms crap in-crap out.
7. And so what does this all mean? I’ll let Wilson have the second to last word: “So what does a test measure in our world? It measures what the person with the power to pay for the test says it measures. And the person who sets the test will name the test what the person who pays for the test wants the test to be named.”
In other words it attempts to measure “’something’ and we can specify some of the ‘errors’ in that ‘something’ but still don’t know [precisely] what the ‘something’ is.” The whole process harms many students as the social rewards for some are not available to others who “don’t make the grade (sic)” Should American public education have the function of sorting and separating students so that some may receive greater benefits than others, especially considering that the sorting and separating devices, educational standards and standardized testing, are so flawed not only in concept but in execution?
LikeLike
Shaggy’s evil testing twin, Mr. Ccraptastic.
LikeLike
“Common Core Curriculum Commissariat College and Career Ready Assessment Program (CCCCCCRAP)” indeed!
LikeLike
“Shepherd’s Pie”
Shepherd’s in the belly
The belly of the beast
Must be really smelly
Digesting testing feast
..but someone has to do it, right?
LikeLike
One of the real tragedies is, that in the process of constraining curricula, in the process of ignoring content knowledge, the Common Core standards have become a subversive tool for limiting possibilities, limiting ideas, and limiting student curiosity and interests.
What parent would ever want to sign up for an educational program that deliberately limited their child’s potential?
LikeLike
Agree
LikeLike
“MAP to nowhere”
I was asked to sit in on this presentation for this assessment in our school. We were “selected” to use it. I had to laugh. We paid Kevin Baird millions of dollars to consult with our districts and principals, and he told us the look fors in curriculum: Two part questions and paired texts. As soon as I saw what the assessment looked like I knew it wasn’t aligned and that the information wouldn’t do a damn thing to inform instruction. So, I said to the presenter what my objections were, and shook my head. The very people who paid Baird and listened to his counsel on curriculum and assessment decisions, are the same people making purchasing decisions that don’t meet those standards.
This test really impacts a school, especially an inner city school such as mine. It is tragic to tell so many families that their child doesn’t possess the literacy skills to pass an exam at the level established by these standards. There has to be a standard and an assessment.
I oft like to ask you, “What do you suggest to replace it?”
Linda
LikeLike
“There has to be a standard and an assessment.”
Says who??
LikeLike
Duane, Every course has standards. Every company has standards. Standards are measured. Schools need to run by a standard. A school can’t rightfully say to a teacher, “Here are your sixth graders. We wish ya the besta’ luck!”
I teach. It’s nice to know what it is my students are expected to know and to teach that and beyond. Standards are helpful. The assessment needs to match the standards. This isn’t always the case, which I believe is part of Bob’s argument.
I say. And that’s enough of an endorsement as I need to share.
Linda
LikeLike
Linda,
The tests don’t need to match the standards. If they do, it encourages test prep. If kids can read, thould be able to demonstrate it on any grade appropriate test
LikeLike
Thanks for the insights. I feel the same way about a “rigorous” GED. Why erect more barriers for people that are aspirational, who can and want to contribute to the economy while they better provide for their families? Also, where’s the fundamental research to prove that people that pass the test will be better workers? All we do by raising the bar on the GED is make more people less able to be self sufficient and more likely to be dependent on government assistance. Lots of people have practical skills, but they will fail a bubble test. What does failing more people accomplish?
LikeLike
Excellent, Bob! Good to see a post from you again. How has this year been going for you? It’s interesting to see what’s been going on in Hillsborough. Bill Gates and MaryEllen Elia certainly made a mess of things there.
LikeLike
I second that, cyn3wulf: It IS good to see Bob Shepherd on here. And, love the ode to “Shepherd’s Pie” by SomeDAM Poet a few spots up. All great.
LikeLike
It would be useful for the New York State Education Department to take a look at Bob’s well articulated objections to the Common Core Standards and the required testing. But even if many of us NY teachers were to try to submit his analysis during this time of the supposed “review of the CC” in NY, as a valid criticism of the CC, it would be tossed out and “not considered.” In other words, NYSED wants to appear as if they are giving everyone a chance to improve the standards and voice their opinions and be heard. But in actuality this is a ruse because after they look at all comments and make whatever small tweaks they may allow then they will announce that all opting out and protests must stop because everyone had a chance to make changes. I can already see the headlines. New Yorkers make peace with the Common Core….now they’re lovin’ it.
Everyone in NY State is being given the opportunity until midnight on November 30, 2015 to take a survey supposedly designed to get feedback about the Common Core so that changes can be made if deemed necessary by NYSED. Here is the link. However, they have set it up so that you cannot comment anonymously and you can only critique each standard for each subject for each grade one by one. If you object to a particular standard you must suggest a specific change to that one standard. General comments will not be considered as stated in the Terms below.
http://aim-high-ny.statestandards.org/
Terms and Conditions
The State Education Department (SED) owns the rights to the feedback application in partnership with Academic Benchmarks.
Any comments (whether anonymous or those users identifiable by email address) will become the rights and property of SED (removing any personal, corporate, or institution copyright entitlement).
Comments must relate to specific, individual standards. General comments regarding the standards must articulate specific feedback about identifiable standards. Comments that do not provide feedback specific to individual standards will not be considered as part of the review process.
LikeLike
Bob Shepherd, thanks for this concise summary– the ‘Short Form’. And thanks for your several beautifully-detailed critiques of CCSS-ELA, citing chapter & verse (collectively, the ‘Long Form’) posted over the last year or two.
BTW, do you have a favorite among the 50 states’ pre-CCSS ELA standards?
My kids got a great pre-CCSS ELA public ed here in NJ. The NJ ELA Core Curriculum Stds, ca. early ’90’s, were blessedly short & open-ended. Can no longer find them online.
Our particular school district got its K-12 LA depts together at some point prior to our ’92 matriculation & developed a scaffolded writing program– the sort of thing a school system can do when not micromanaged by the state. The town’s hs grads became known for superior writing ability. My boys were music-techies (hardly book-&-pencil types); they attended medium-rung NY & NJ colleges w/tech specialties. All 3 noted w/surprise that their writing skills were significantly better than those of their peers. Don’t know whether the district is able to sustain this now that they are encumbered w/the various assessments associated w/Marzano & CC-ELA (implemented 2013).
LikeLike
PS– that’s me, Sp & Fr Freelancer. Since I’ve joined WordPress my moniker switches at whim.
LikeLike
Great piece, Robert. And, as usual, “right on the money”.
Ellen Klock
LikeLike
Reblogged this on christybez and commented:
Must read
LikeLike
Is the ultimate goal to create two classes – the college bound (1/3) and the failures (2/3) who won’t be able to graduate from high school or get a GED?
If NYS continues to require high school students to pass 5 Regents Exams and 2 or more of those exams are Common Core Regents (ridiculous in content and designed to fail most students), then we really will need to raise the minimum wage to $15 an hour because the majority of workers will be unable to qualify for professional positions.
LikeLike
Thank you Bob Shepherd and Diane Ravitch! hope this is shared with Commissioner Elia (NYSED and Board of Regents) and Tim Kremer (NYSSBA) to share with all. “The test questions therefore tend to be tricky and convoluted.” I also want to say the 5th grade modules are test prep created exactly to be tricky and convoluted such as you stated in the above. The modules last all year with fiction/non-fiction literature or non-fiction articles that have inference questions throughout. The middle and end assessments for each module consist of multiple choice and short response questions, and essays. It is horrible to watch students flip pages back and forth after they have close read, annotated text, and then must cite evidence, paraphrase or summarize. I have had several adults take these only to ask what the point is to have such subjective questioning on them.
Fifth grade students are ten year olds. By the time they are done taking the tests, which the module says should take no more than 45 minutes, but take some a great deal more than that if written the correct way and checking over, the students are exhausted and very little can be taught the rest of the day. Many times they look at me and are confused with questions asked. I agree with them as many of the questions are ridiculous and could have more than one answer, but the modules ask for the BEST answer as the state tests do.
Just think what wonderful ways we can (and do as a silent revolution) Bloom’s Taxonomy, Literature Circles, and much, much more.
What a waste of time when we use modules all year long and then still have 70% of our kids fail, yes fail. But our kids aren’t failing anyone. Our teachers aren’t either. It’s Governor Cuomo and his love of Common Core and the billionaire’s who think this is a wonderful way to make everyone common. Well, guess what? Diversity is here to stay in my classroom! No robots for us. If you read this and haven’t refused the tests yet, do so now at http://www.nysape.org. My kids will always be taught learning takes place everywhere, especially beyond the classroom walls! Watch out Governor Cuomo, the teachers have awoken and are finally speaking out! Don’t forget, Commissioner Elia is the one who gave us permission to do so!
LikeLike