When I was in San Francisco, I talked about SLAYING GOLIATH with Susan Solomon, president of United Educators of San Francisco. It was videotaped by CSPAN Book TV and has been broadcast.
Here is the full interview:
https://www.c-span.org/video/?468918-1/slaying-goliath
Now that most public gatherings have been canceled, I am happy to share this conversation with you.
Please let me know what you think about the discussion. I appreciate your feedback.
If you read the book and like it, please do me the great favor of giving a copy to a local school board member and/or your state legislator.
The way to improve public education is to educate the public.
Two important chapters in SLAYING GOLIATH that you should pay attention to: Why standardized testing preserves the achievement gap (it is built into the design); and what cognitive scientists in the 21st century have learned about the sources of motivation.
Standardized tests are normed on a bell curve. The bell curve never closes. Advantaged kids dominate the top half. That’s true of every standardized test.
Diane Ravitch's blog 
I am glad you specifically point out these two chapters in the book. It struck me that these chapters, which explain why the disruption was always futile as a method of school improvement, were crucial.
What it made me think of is the necessity of being short and to the point when we make a case against disruption methods. When A Nation At Risk became common currency, few short phrases were all people knew about it.
I think I will try to place those two chapters Ina short summary (as if they already are not) that even an administrator would take the time to read and understand.
Thank you. The achievement gap never closes because standardized tests are normed onto a bell curve. By design, the bell curve never closes. Advantaged kids dominate the top half. Kids who lack those advantages dominate the bottom half. That is the way it is in the results of every standardized test.
The dirty secret of the standardized testing industry is the breathtakingly low quality of the tests themselves. I worked in the educational publishing industry at very high levels for more than twenty years. I have produced materials for all the major standardized test publishers, and I know from experience that quality control processes in that industry have dropped to such low levels that the tests, these days, are typically extraordinarily sloppy and neither reliable nor valid. They typically have not been subjected to anything like the standardization procedures used, in the past, with intelligence tests, the Iowa Test of Basic Skills, and so on. The mathematics tests are marginally better than are the tests in ELA, US History, and Science, but they are not great. The tests in English Language Arts are truly appalling. A few comments:
The new state and national standardized tests in ELA are invalid
First, much of attainment in ELA consists of world knowledge–knowledge of what–the stuff of declarative memories of subject matter. What is metonymy? What are the parts of a metonymy? How does it differ from metaphor and synecdoche? What was American Gothic? What are its standard motifs? How was it related to European Romanticism and Gothic literature? How did it differ? Who were its practitioners? What is terza rima? What is dactylic hexameter? What is deconstruction? What is reader response? the New Criticism? What does it mean to begin in medias res? What is a dialectical organizational scheme? a reductio ad absurdum? an archetype? And so on. The so-called “standards” being tested cover ALMOST NO declarative knowledge and so miss much of what constitutes attainment in this subject. Imagine a test of biology that left out almost all world knowledge and covered only biology “skills” like–I don’t know–slide-staining ability–and you’ll get what I mean here. This has been a MAJOR problem with all of these summative standardized tests in ELA since their inception. They don’t assess what students know. They test, supposedly, a lot of abstract “skills,” but they don’t even do that.
Second, much of attainment in ELA consists in procedural knowledge–knowledge of what to do. The “standards” being tested define skills so vaguely and so generally that they cannot, as written, be sufficiently operationalized,to be validly tested.
Third, nothing that students do on these exams even remotely resembles what real readers and writers do with real texts in the real world. When people read in the real world, they do so to find out what the author has to say about a topic or in order to have a vicarious experience. The CCSS testing on trivialities related to texts thus sends the whole process of teaching kids to read and pointing them to great texts from the canon off in the wrong direction. Because they don’t deal with real reading and response, ipso facto, these tests cannot be valid tests of actual reading and writing ability. We do not live in a world of writing done in five-paragraph themes.
Fourth, standard standardized test development procedures require that the testing instrument be validated. Such validation requires that the test maker show that the test correlates strongly with other accepted measures of what is being tested. No such validation has been done for the state and national standardized ELA tests. None. And, given the vagueness of the “standards,” none could be. Where is the independent measure of proficiency on Common Core State Standard ELA.11-12.4b against which the items on the state and national measures have been validated? Answer: There is no such measure. None. So, the tests fail to meet a minimal standard for a high-stakes standardized assessment–that they have been independently validated.
The test formats are inappropriate
The state and national tests consist largely of objective-format items (multiple-choice and so-called evidence-based selected response items, or EBSR). On these tests, such item formats are pressed into a kind of service for which they are, generally, not appropriate. They are used to test “higher-order thinking.” The test questions therefore tend to be tricky and convoluted. The test makers, these days, all insist on the answer choices all being plausible. The student is to choose the “best” answer from among a list of plausible answers. Well, what does plausible mean? It means reasonable. In other words, reasonable answers are wrong answers. So, the test questions end up being extraordinarily complex and confusing and tricky, all because the “experts” who designed these tests didn’t understand the most basic stuff about creating assessments–that objective question formats are generally not great for testing higher-order thinking, for example, and are best reserved for testing straight recall. The use of these inappropriate formats, coupled with the sloppiness of the test-creation procedures, results in question after question where there is, arguably, no correct answer among the answer choices given or one or more choices that are arguably correct. Often, the question is written so badly that it is not, arguably, answerable at all given the question stem and text provided. I did an analysis of the sample released questions from a recent FSA ELA practice exam, issued by the test maker, and demonstrated that such was the case for almost all the questions on the exam, so sloppily had it been prepared.
The tests are diagnostically useless
Many kinds of assessment—diagnostic assessment, formative assessment, performative assessment, some classroom summative assessment—have instructional value. They can be used to inform instruction and/or are themselves instructive. The results of these new national and state standardized tests are not broken down in any way that is of diagnostic or instructional use. Teachers and students cannot even see the tests to find out what students got wrong on them and why. So the tests are of no diagnostic or instructional value. None. None whatsoever.
The tests have enormous opportunity costs
I estimate that, nationwide, schools are now spending a third of the school year preparing students to take these state standardized tests. That time includes the actual time spent taking the tests, the time spent taking pretests and benchmark tests and other practice tests, the time spent doing test prep materials, the time spent doing exercises and activities in textbooks and online materials that have been modeled on the test questions in order to prepare kids to answer questions of those kinds, and the time spent on reporting, data analysis, data chats, proctoring, and other test housekeeping. That’s all lost instructional time.
The tests have enormous incurred costs
The U.S. spends about 1.7 billion on contracts for state standardized testing alone. The PARCC contract by itself was worth over a billion dollars to Pearson in the first three years, and you have to add the cost of SBAC and the other state tests (another billion and a half?) to that. No one, to my knowledge, has accurately estimated the cost of the computer upgrades that were (and continue to be) necessary for online testing of every child, but those costs probably run to 50 or 60 billion. This is money that could be spent on stuff that matters—on making sure that poor kids have eye exams and warm clothes and food in their bellies, on making sure that libraries are open and that schools have nurses on duty to keep kids from dying. How many dead kids is all this testing worth, given that it is, again, invalid as assessment and of no instructional value?
The tests dramatically distort curricula and pedagogy
The tests drive how and what people teach and much of what is created by curriculum developers. These distortions are grave. In U.S. curriculum development today, the tail is wagging the dog. To an enormous extent, we’ve basically replaced traditional English curricula with test prep. There’s enormous pressure on schools to do this because everything–school and individual evaluations, for example, depends upon the test results. I won’t go into detail about this, here, but this is an ENORMOUS problem.
The tests are abusive and demotivating
Our prime directive as educators should be to nurture intrinsic motivation—to create independent, life-long learners. The tests create climates of anxiety and fear. Both science and common sense teach that extrinsic punishment and reward systems like this testing system are highly DEMOTIVATING for cognitive task. See this:
https://search.yahoo.com/search?fr=mcafee&type=C111US662D20151202&p=daniel+pink+drive+rsa
The summative standardized testing system is a really, really backward extrinsic punishment and reward approach to motivation. It reminds me of the line from the alphabet in the Puritan New England Primer, the first textbook published on these shores:
F
The idle Fool
Is whip’t in school
The tests have shown no positive results
We have been doing this standards-and-testing stuff for more than a decade and a half now. Richard Rothstein, the education statistician, has shown that turning our nation’s schools into test prep outfits has resulted in very minor increases in overall mathematics outcomes (increases of less than 2 percent on independent tests of mathematical ability) and NO IMPROVEMENT WHATSOEVER in ELA. Simply from the Hawthorne Effect, we should have seen some improvement. And that suggests that all this testing has actually decreased outcomes, which is consistent with what we know about the demotivational effects, for cognitive tasks, of extrinsic punishment and reward systems. It’s the height of stupidity to look at a clearly failed approach and to say, “Gee, we should do a lot more of that.”
The test makers are not held accountable
All students taking these tests and all teachers administering them have to sign forms stating that they will not reveal anything about the test items, and the items are no longer released, later, for public scrutiny, and so there is no check whatsoever on the test makers. They can publish any sloppy crap with complete impunity. I have done private analyses of sample release questions from these tests, and those are devastating for the test makers, but even these I cannot release for fear of being sued by Pearson and the other test-making scam artists.
In state after state in which the new generation of standardized tests has been been given, we have seen enormous failure rates. In the first year, fewer than half the students at New Trier, Adlai Steven, and Hinsdale Central–the best public schools in Illinois–passed the new PARCC math tests. In New York, in the first year of PARCC, 70% of the students failed the ELA exams and 69% the math exams. In New Jersey, 55% of students in 3-8 failed the new state reading test, and 56% the new math test. A couple years ago, Florida delayed and delayed releasing the scores for its new ELA and math exams. Then they announced that they weren’t going to release only T-scores and percentiles but were still working on setting cut scores for proficiency. LOL. Criterion-based testing, as opposed to norm-referenced testing, is supposed to set absolute standards that students must meet in order to demonstrate proficiency. I suspect that what happened in Florida–the reason for the resounding silence from the state–is that the scores were so low that they couldn’t set cut scores at any reasonable level without having everyone fail.
https://www.c-span.org/video/?468918-1/slaying-goliath&start=277
This is the link to the whole presentation. There is also a short clip, less than five minutes, made by a viewer. Highlights of the book are well-presented and to an apprciative audience.
I ordered a book earlier today for a school board candidate! You are wonderful, Dr. Ravitch. Thank you from a Texas Kids Can’t Wait!
Thank you, Bonnie. I was disappointed that I had to cancel my Texas trip.