David Berliner and two colleagues wrote an article proposing a simple and research-based way that public schools can save millions of dollars annually: Stop testing every student every year. Test every third year or every other year. They explain why this makes sense in an article posted on Valerie Strauss’s blog The Answer Sheet.
Strauss begins:
States spend millions of dollars every year to purchase standardized tests in an exercise that has come under strong criticism in recent years for reasons including the quality of the exams and the often invalid ways that districts and states use the scores.
While the billion-dollar testing industry is undergoing changes, with a bigger share of its spending going to the purchase of digital exams, the same questions remain, including: Are states wasting money?
The federal government requires annual statewide tests in reading/language arts and mathematics for all students in grades three through eight and once in high school, and some states tack on other standardized exams. A decade ago, one analysis found that states spent a combined $1.7 billion on these exams, and experts say the total has only gone up.
This post argues that the states are wasting money, and it explains an alternative to save money and increase instructional time. It was written by David C. Berliner, Norman P. Gibbs and Margarita Pivovarova.
Berliner, Regents’ professor emeritus at the Mary Lou Fulton College of Education, is a past president of the American Educational Research Association who has published extensively about educational psychology, teacher education and educational policy. Gibbs is a program evaluator for the Mesa Unified School District in Arizona whose research focuses on assessment and accountability, comparative and international education, and inclusive and participatory decision-making. Pivovarova is an associate professor in the Mary Lou Fulton Teachers College at Arizona State University whose research focuses on the relationship between student achievement, teacher quality and school contextual factors.
This post argues that the states are wasting money, and it explains an alternative to save money and increase instructional time. It was written by David C. Berliner, Norman P. Gibbs and Margarita Pivovarova.
Berliner, Regents’ professor emeritus at the Mary Lou Fulton College of Education, is a past president of the American Educational Research Association who has published extensively about educational psychology, teacher education and educational policy. Gibbs is a program evaluator for the Mesa Unified School District in Arizona whose research focuses on assessment and accountability, comparative and international education, and inclusive and participatory decision-making. Pivovarova is an associate professor in the Mary Lou Fulton Teachers College at Arizona State University whose research focuses on the relationship between student achievement, teacher quality and school contextual factors.
By David C. Berliner, Norman P. Gibbs, and Margarita Pivovarova
Could state educational policymakers do with a few million extra dollars? Surely, America’s teachers can help us all think of something to do with that money. We know how they can do it.
We explain below how this is done, as we did more extensively in a just-published article in Education Policy Analysis Archives, a respected, peer-reviewed educational research journal.
We presented data suggesting a remarkably easy and substantially cheaper way for each state to get the information it desires about the academic performance of its schools from the standardized tests it uses. In addition, following the advice offered in this article, there would also be an increase in instructional time for students. Let us set the stage for this research first.
Suppose a set of nonidentical triplets are identified at age 5. One is tall for his age, one is of medium height, and one is short for his age. At age 6, what is the chance that these children have changed the order of their heights? Sure, they will probably be a little taller, but the order is highly likely to be the same, almost every year. Certainly, if one of the triplets takes special hormones, or one contracts a lengthy disease, the order might change. But without an unusual event, these triplets are quite likely to grow into adulthood as they were — one relatively short, one medium, and one tall. Their rank order, not their height itself, will almost assuredly remain the same.
If we used statistics and did year-to-year rank order correlations for the triplets’ height, the result would likely be a correlation of 1.00, indicating a perfect correlation. This would inform us that the rank order of the triplets is always the same, even if their heights do actually change a bit until they are well past puberty. But even then, regardless of their actual height, their relative height is likely to be constant, and thus it probably need not be measured frequently at all. We “know” that year after year, when we measure their heights, the triplets are almost assuredly still going to be tall, medium, and short in comparison to each other, Eventually, it simply wouldn’t be worth the effort to measure their heights frequently.
Well, it turns out that the hundreds of schools in a state line up in scores just as do as the triplets. Their relative test scores — whether low, medium or high — barely change at all, year after year, regardless of the scoring system used by the standardized testing company. If the relative scores don’t change much year after year, except under some unusual circumstances, why would you need to test the students in those schools to learn how they are doing, year after year?
Here, for example, are the correlations between test scores in mathematics, from one year to the next, for every elementary school in Nebraska, for the years 2014 to 2018. Those year-to-year correlations are .93, .95, .94, .90, .95. These data inform us that if you know this year’s scores in mathematics for each Nebraska school, you know almost perfectly how those schools will test the following year. It’s the equivalent of knowing the order of the heights of the triplets this year, and thus being quite sure you would know the order of their heights were you to measure them the next year. Similarly, if you already know the standardized test scores for every elementary school in Nebraska, you don’t really need to test the next year. Next year’s ordering of Nebraska’s schools will look very much like this years’ ordering of its schools. So why not skip a year or two of testing, and save millions of dollars and millions of instructional hours?
With correlations in the .90’s between last year’s test scores and this year’s test scores, as was empirically obtained, you certainly don’t need to test every year to know how the schools in Nebraska are performing. If big changes in a school’s performance did occur, you’d certainly pick that up through testing every other year. Apparently, unless a schools catchment area changes, or is rezoned so it has a big shift in population, or it must deal with a natural (earthquake) or man-made disaster (a school shooting) that upends the school community, a school’s standing in a pool of standardized test scores will not change much from year to year.
We repeated our analyses in another state, at other grade levels, and for other subject matters. For example, here are the correlations for one year’s standardized achievement test scores in reading, with the following years’ achievement test scores in reading, for all of Texas’s middle schools, over five years: .92, .91, .91, .93, .93. As in Nebraska, knowing this year’s standardized test score informs us almost perfectly what next year’s test score will be. We know how each school will perform because of its previous score. The rank order of a school, vis-a-vis every other school in the state, is quite stable. Mandated achievement tests in Nebraska and Texas need not be given every year to answer the question: How is this school doing? Testing every other year in Nebraska and Texas, and we suspect in all other states, would yield the same information desired by those concerned about how the schools are doing academically.
But it gets better, and thus even more millions of dollars might be saved! Presented next are the correlations between tests of reading given two years apart on Texas’s middle school reading test (.89, .89, .89, 90). And here are the correlations between tests of reading given two years apart for Nebraska middle schools (.92, .95, .91, .97). In other words, almost the same rank order of schools will be present in Nebraska and in Texas if you tested every third year, saving the states a gazillion dollars in money and time, and it would also reduce the annual surge in the test anxiety of thousands of U.S. students, teachers, and parents.
Testing every third, or every second year, results in virtually no loss of information for district, state or federal agencies. We are not recommending doing away with the assessment of student achievement by means of standardized achievement tests, but we are pointing out that we seem to have overdone it. Testing annually eats up a great deal of instructional time and a large amount of money but yields little new information for states, districts and schools.
To those who say that “the teachers need the standardized test results to know how their students are doing,” we have two answers. First, experienced teachers already know how their students are doing in relation to their states’ recommended curriculum, and they don’t need a standardized test to provide them with that information. Research evidence informs us that experienced teachers are quite good at predicting the rank order of each of their students on their own states’ standardized achievement tests.
The other answer to this tired rationale for standardized testing is related to scheduling. The tests are typically given in spring. Test results are, therefore, usually analyzed over the summer months. Test results, by necessity, are given back in the fall of the calendar year, to teachers who have already passed their students on to teachers in the next grade! The information about student achievement, when teachers no longer have those students, comes too late to make any midcourse corrections in their instruction.
And some have argued that achievement testing has value for school administrators, who might then be able to identify exemplary and ineffective teachers from the test performance of the students those teachers had the previous year. But that is no easy identification to make, since each year’s classroom level achievement test data is greatly affected by the kinds of students a teacher was assigned. Substantial differences in achievement test scores occur for teachers depending on the numbers of second-language learners, or students with high absentee rates or special-education students who were assigned to their classrooms. In fact, even classes with slightly more girls than boys generally score higher on tests than classes with more boys than girls. So, inferring teacher competence from standardized test results is quite problematic.
Now that this research article has been published in a peer-reviewed journal, we wonder which state will be first to petition the federal government for a waiving of the current testing requirements? Will the federal government grant such waivers, or are its policies immutable? We are pretty sure that a state choosing to test every third year, or every other year, will save millions of dollars and millions of instructional hours, with no loss of the information it believes to be useful. A reconsideration of our nation’s assessment policies is surely warranted.
Exactly. Utah makes the 9th and 10th graders take the ACT Aspire every year, even though it’s not federally mandated. We NEVER s e the scores, nir do the students. The state says it “helps students prepare” for the mandated ACT in 11tg grade, but how can it when no one ever sees the scores? It ruins 2 full school days of instruction for all of the middle and high school grades as everything has to be rearranged. Don’t get me started on how much the Big Standardized Tests (thank you, Peter Greene) take over everything, make it impossible for students to use libraries for 1/4 of the year, and waste so much money.
No, not exactly!
“Test every third year or every other year. They explain why this makes sense”. That’s exactly what you want?
First, it doesn’t make sense at all when the fact that all standardized tests are onto-epistemologically invalid and cause harm to students.
Second, these bullshit (BS as named by Greene) tests do not help any student learn anything as the results of the tests, i.e., an item by item review by the student and the teacher, IS NOT ALLOWED. Students are not allowed to challenge their score and/or grading of the test.
Third, the amount of time, effort and monies thrown into the shitpot that is standardized testing could be better spent on things that really matter, ya know like lower class sizes, more student oriented/desired class offerings-arts, music, second languages, special ed needs that are not the subject matter covered by those invalid tests, and/or counseling services.
The ONLY logical and ETHICAL choice is to eliminate all the standardized test that are not specifically diagnostic for a particular student’s needs, give the time and resources (by that I mainly mean proper amount and trained personnel) back to the classroom teachers to properly implement a teaching and learning process that is to the benefit of the students.
You’re right of course, Duane, but the powers that be won’t condone getting rid of standardized testing completely. At this point, fewer tests would be better.
I understand what you are saying. . . but some of us have to take the “no compromise” approach. At least I don’t and won’t compromise.
Too much harm is still being caused to too many students for me to compromise on this.
Duane, I’ve forgotten whether I’ve asked you this before, but do you feel the same way about the old Iowa Basics [or Stanford] grade-span tests? They were strictly diagnostic— no effect on grades, schools, teachers. In the ‘90s/ early 2000’s, my kids took them around 2nd-gr & 5th-gr & once more in midsch.
The tests covered the 4 main subject areas. For one son in particular (the 1 of 3 who didn’t have an IEP 😉), they showed where his academic strengths/ interests lay. [That info didn’t necessarily synch with school grades, which tended to be mostly about “did you do what you were supposed to do.”] Which helped in selecting midsch courses.
Comparing those 3 scores across a 6-yr span—vs school grades– then illustrated that his performance was steadily declining in two subjects where he’d shown strength & interest early on. I figured that meant he needed a different approach, & eventually convinced him to try our hisch’s school-within-a-school project-based learning program [ongoing since late ‘70’s]. Which was a smashing success. I don’t think I would have understood the issue without those no-stakes, grade-span tests.
If the makers of the Iowa Test of Basic Skills (ITBS) claimed it was a diagnostic test to assess for supposed deficiencies in learning then that is different than the standards and testing malpractice regime against which I rage. They still have many of the invalidities that Wilson points out (almost all assessments do have some validity issues) but how one approaches a testing situation is what distinguishes different standardized tests. I’ve not looked that closely at what the ITBS claims nor how the test is developed, administered, scored and then what is done with the results. Again, if the process is open and honest to its limitations and those are communicated with the parents so that all understand, it can be an acceptable process.
No its not what i would want either–but the data I was able to obtain led me to believe that we could even get the test lovers to reduce the amount of testing that we do. It would be a step in the right direction.
“No its not what i would want either–but the data I was able to obtain led me to believe that we could even get the test lovers to reduce the amount of testing that we do. It would be a step in the right direction”
I think it’s important to have sides that represent the extremes as well as those who tend towards the middle.
Considering the track record of the last 2+ decades, I think a compromise is more realistic than demanding a complete abandonment of the test industry’s investment.
Here’s a better idea.
Don’t test at all.
Instead assess whether students are meeting targeted learning goals and use that information to help students learn instead of generating grades that only serve to grant or deny access to privilege.
How to do this is described in the book — Knowing the Learner: A new approach to Educational Information.
Paul Zachos, PhD
Director, Research and Evaluation
Association for the Cooperative Advancement of Science and Education (ACASE)
110 Spring Street Saratoga Springs, NY 12866 |
paz@acase.org | http://www.acase.org
Thank you for this insightful post. Reflecting on the past sixty years of education, I concur with Berliner’s conclusions. We are wasting time, money and the well-being of our young people by over testing them and holding on to the false belief that punitive, high stakes testing is somehow going to magically improve outcomes. It is almost as absurd as believing that privatizing schools while subtracting dollars from public schools will somehow be a magic bullet for “success” for everyone.
Nobody wins when we demoralize a generation of students. Young people today are at greater risk of depression and suicide, and over testing, while not the sole cause, is a contributing factor. When I hear that parents are abandoning public schools because of a robotic curriculum, driven by the pursuit of data collection, it makes me feel sick because I have derived a great deal of knowledge and joy from both learning and teaching in public schools. Testing is not the primary mission of education. We have lost our way.
Politicians under the influence of billionaires and corporations have hijacked public education by imposing a “reign of terror” on public schools, students and teachers. The whole deform movement is educationally unsound and unhealthy. Politicized education that impedes learning and tries to turn it into a commodity is harmful to our young people and our foundational public institution, public education. Our public schools primary mission is to serve the needs of students, not outside political or commercial interests.
Anyone that follow education long enough will see history repeat itself. Berliner’s three year cycle of testing is exactly what New York did when I started teaching there in the 1970s. Standardized tests were given in grades 3, 6 and 8. High schools had their own testing schedules, but there was no annual testing and no high stakes attached to tests. Students are not widgets, and schools are not businesses. Schools are a collection of people in communities. Business practice is not necessarily appropriate for education. We need to stop acting as though schools are corporate structures. It is time to scale back our testing obsession, which has taken on a life of its own, for no better results, and it has also become more noxious than helpful.
rt– “When I hear that parents are abandoning public schools because of a robotic curriculum, driven by the pursuit of data collection, it makes me feel sick”
It makes me feel especially sick because I am convinced there’s a method to that madness, and it’s about crushing the natl pubschsys. It can’t be an accident that NCLB was implemented in the early bloom of the charter movement. Or that Obama admin doubled down with RTTT & Common Core while ramping up the CSP [& meanwhile state voucher programs were getting off the ground].
Chew on this:
ALL standardized tests are invalid and a bastardization of the teaching and learning process and using the results for anything is, as Wilson puts it, “vain and illusory” or as I say complete bullshit.
Take a bite…
Credentials are only as good
as the SCORESthey rest on.
Then those credentials are worthless.
Wilson is a crackpot on the level of people with high-school educations who write physicists to tell them why Einstein was all wrong and their new Quantum Space Wave Theory Based on the Pythagorean Theorem is the Ultimate Theory. The federally mandated tests are garbage, but not for these reasons, and suggesting otherwise just leads people, erroneously, to think that all folks who oppose the tests are crackpots. I really wish that this would stop so that we can have conversations about the ACTUAL issues with the tests that render them invalid.
Your evaluation of Wilson is wrong. Just because you can’t understand what he has shown doesn’t mean that he is a crackpot. What are the ISSUES you have with Wilson’s analysis? Be specific, not some generalized claptrap like your wrong thoughts above.
Be that as it may, what are the ACTUAL issues with the tests? Lay them out for us, please!
Duane, I am not going to go back and rehash this. When you first started posting about Wilson years ago, I looked up his book. I started reading it. I got about a third of the way through and said to myself, this guy is totally looney tunes. I am not going to waste my valuable time going back and doing a detailed, page-by-page analysis of this nonsense for the same reason that I am not going to go spend my time writing about any other 1,000-page set of ravings by random nutsos–the Unbomber Manifesto, Ignatius Donleavy’s Atlantis: The Antideluvian World, Christopher Harper’s My Manifesto, Utter waste of my time and energies.
But let people judge for themselves. I encourage them to go start reading the Wilson monograph so that they can see just how freaking weird it is.
Here, the actual reasons why these tests are not valid:
https://bobshepherdonline.wordpress.com/2020/03/19/why-we-need-to-end-high-stakes-standardized-testing-now/
“I got about a third of the way through”
Get back to us after you’ve read and understood it completely. Until then your analysis is “vain and illusory.”
I’m sorry. The profound wisdom of Wilson seems to be beyond my mere moral comprehension. LMAO.
LOL
–He who is in the grips of vain illusions. Haaaaaaa!
cx: mortal comprehension
http://artastes.blogspot.com/2006/01/schizophrenic-art.html
The idea that no mental attainments can be measured is simply looney. I have addressed this, here, many times. I have explained how I can easily put together a test that will determine whether a person knows his or her multiplication table from 0 x 0 to 10 x 10 or what percentage of Level 1 Kanji he or she knows. Not going to go back over that again.
I’ll take another bite. One of the issues that Wilson harps on is that the term measurement refers to physical quantities with agreed-upon scales, like the measurement of distance in meters but that educational measurement isn’t like that. This is a totally specious argument of the kind that one often finds in the writings of schizophrenics. I am not saying that Wilson is a schizophrenic, just that his monography reads, to me, as very similar to the writings of schizophrenics. So, here’s an example of what I’m talking about, of that kind of insane argument:
When you walk along a sidewalk, you are moving. So, moving is walking. Therefore, flying is impossible. Cannot be done. The belief in flying is a mass delusion.
Same form as Wilson’s argument about measurement. One of the issues with that manner of thinking is referred to as the fallacy of composition.
Also, straw man. Wilson asserts that incidental characteristics of measurements are essential–definitive of measurement. Then, having set up a straw man–a spurious definition of that which is not measurement–he disallows lots of kinds of measurement that people do in fact validly do, including measurement by standardized testing of things like knowledge of Level 1 Kanji or the names of the twelve optic nerves.
So, for example, Wilson claims that educational measurement doesn’t exist because measurements require an external measurement standard, but this is BS. Simple counting, for example, is one kind of measurement that doesn’t. Let’s see, let’s measure our new hire rate. We had one in Jan, 3 in February, 10 in March, . . . OK, divide that by 12. That’s our average yearly monthly hire rate. we have MEASURED it. No external reference scale required.
No you haven’t measured it, you’ve counted it and figured a per monthly rate. You’re wrong on this aspect.
Lol. You and Mr. Wilson cannot simply decide that the word “measurement” has only the narrow meaning that you ascribe to it.
Words have multiple meanings, Duane. Electrical potential is not the same as a basketball team’s potential, but the two different meanings share some PART of their meaning in common.
This should not require explaining.
One can’t simply decide that “glory” means ” a nice knock-down argument” and then base a whole “educational theory” on this. LOL.
“I got about a third of the way through”
Get back to us after you’ve read and understood it completely. Until then your analysis is “vain and illusory.”
Has it occurred to you, Duane, that a unit of measurement does not have to be an external reference scale but, rather, an object or fraction of an object?
We even use the term “measurement” for qualitative judgments (e.g., “take the measure of the man” in the sense of “evaluate him”).
Where Wilson completely goes off the rails is in deciding that because measurement must means what he thinks it means, ALWAYS, that we cannot measure things like whether someone has learned the times table or can name the optic nerves in order. But we in fact give tests to doctors and optometrists to MEASURE knowledge of the latter, and we in fact give tests to kids to MEASURE knowledge of the former. And any claim to the contrary (there cannot be ANY valid standardized test of ANYTHING) is just bat___ crazy and demonstrably false, and everyone knows this because they live in a world where this kind of thing is validly done ALL THE FREAKING TIME.
“I got about a third of the way through”
Get back to us after you’ve read and understood it completely. Until then your analysis is “vain and illusory.”
Some men have wives. It does not follow from that that Randy Rainbow, who does not have a wife, is not a man. Some measurements use external reference scales. It does not follow from that that all measurements do.
I, for one, would prefer that the engineers who designed the airplane and its parts that I am riding in have passed standardized engineering examinations. LOL. To dismiss all examination for intellectual attainment as spurious because it is not “actually measurement” is bonkers. And it makes those of us who oppose the invalid state tests for reality-based reasons look like we belong to some lunatic fringe.
OK Bob now that I see from your 1st link that you have looked into the Iowa Basics, please give me your take on the Q I asked Duane above [@ 2/20 5:34pm].
Ginny, I TOOK the Iowa test a few times when I was a kid. I vaguely remember a lot of grammar questions. But I’m no expert. In generally, if the test covers a CLEARLY DEFINED SPACE and does so adequately, in language at grade level, and deals with knowledge as opposed to broadly drawn, vague skills, AND it has a purely diagnostic purpose, then it will be useful. My take. What’s yours? But I would have to look at some of the Iowa tests again to evaluate them.
Your language learning suggests to me a good analogy for the currently mandated state ELA tests. Suppose that I gave you a “test of mastery of the languages of the world.” One question asked where France was located. The next asked for the meaning of Blin in Russian. And so on, one question per “language” for, say, 40 of the world’s roughly 7,100 languages. Suppose that I then claimed that this was a test of level of proficiency in all the languages of the world and that 26 correct indicated proficiency in this. The ELA tests are PRECISELY like that. A few random questions that are supposed to reveal proficiency in an enormously broad range of very broadly drawn areas of ability.
This ought to be obvious to anyone who has actually looked at these tests and what the makers of them purport to be testing. They are laughable, absurd, ridiculous.
But read the news, and so-called journalists who haven’t looked at all deeply into this report the “results” from these stupid tests as though they were gospel brought down from the mountain. That’s shameful.
Bob, RE: your first link above: This is one of my favorites of your blog posts, & happy to find it/ read it again. Haha, I am especially enjoying your challenge to write 1 mult-choice Q to address the std on researching a word’s pronunciation/ clarification of meaning/ part of speech/ etymology.
One of my joys in retirement is exercising my Span & Fr lit reading/ speaking abilities daily. Only a word-nut like ourselves could enjoy saying it all out loud, checking the inscrutable dictionary symbolism on pronunciation, & not only meaning, but separately, etymology [for me a key to remembering meaning]– & periodically, I find that a confusion on part of speech clouded my understanding.
My answer to Coleman/ Gates: this is an essay question!
People write book-length treatises on each of these all the time! Wonderful to hear about your language studies, Ginny!!! Love to you and yours.
I meant that I am no expert on the Iowa Test of Basic Skills. I used to be quite familiar with it, but I haven’t studied it carefully in a long time. However, I do know a lot about testing. I’ve done a lot of test development over the years, and I’ve studied the process.
So, Duane, you are claiming, based on the work of Mr. Wilson, that ALL credentials received via testing are worthless? So, the fact that someone has passed the Bar Exam is irrelevant to demonstration of the necessary knowledge to practice law effectively? Same for credentialing examinations of potential dentists, doctors, nuclear engineers, civil engineers, pharmacy technicians, and so on.
That’s quite the claim. And yet you assert that this isn’t bonkers?
All of these are based on standardized testing of intellectual attainments, i.e., acquisition of knowledge-and-procedures.
Bob,
Thank you for taking the time to explain your skepticism of Wilson so thoroughly.
I find it very helpful. Like you, I understand and agree how INVALID the state tests are.
But if parents and the public believe the opposition to state tests is coming from folks who appear to want to end all testing for anything because all tests requiring a person to demonstrate any knowledge or expertise are invalid period, then it just seems to hurt and not help.
Plus I really don’t understand how a classroom test devised by a teacher is any more valid than any other test. Nothing is perfect, “testing” will always have flaws, but never testing has a lot more flaws, in my opinion.
Here’s what it boils down to. Some things are credibly testable. Some are not. One cannot, for example, ask ONE question about ONE figure of speech and conclude on the basis of this that a student does or does not have the ability to make sense of passages containing figures of speech or, but this is PRECISELY the kind of thing that the ridiculous state ELA tests do. And one certainly cannot based on ONE question related to the standard concerning making inferences from texts that students are able to make proper inferences IN GENERAL from texts IN GENERAL. This is like saying that you understsand naming conventions worldwide because you know one person named Bob. LOL. It’s idiotic.
In general, in order for something to be validly testable, there must be an objective correlative. This can happen when one is testing clearly identified pieces of knowledge and clearly identified, step-by-step standard procedures and processes. It can also happen when one is testing clearly delimited areas of abstract, axiom-based, a priori material, as in Math and Logic. Otherwise, nope. The idiotic bullet list cobbled together by Coleman et al. based on the terrible state “standards” contains extraordinarily broad statements that cannot be represented by concrete, objective correlative questions in a 30-item test. That this is not obvious to people means that they have not thought at all clearly about these tests and what the makers of them purport that they do.
Oh, and thanks, NYC PSP!
If you have a situation in which knowledge of the answers to a test is objectively, demonstrably equivalent to (or closely equivalent to) or is a reasonable sampling of some set of knowledge of the world, then you are looking at valid testing. The federally and state-mandated tests are laughable on this criterion alone. They should have long ago been laughed off the national stage by people who know better. That they haven’t is more than a little shocking. But there are other profoundly serious problems. The ELA tests, for example, are close to content free, but attainment in ELA, including ability to read, clearly requires a lot of content knowledge. And, ofc, the lower-level state Math tests purport to test abstract reasoning ability when kids at these ages are extremely concrete thinkers. The tests are bs. Really, seriously, it’s time that teachers and administrators and MOST IMPORTANTLY teacher’s unions stood up and said, no more. We are not administering these any more. The unions could stop this bs tomorrow. Until they do that, they are complicit in child abuse and in the wholesale devolution of U.S. curricula and pedagogy.
As Matthew Davis and E.D. Hirsch, Jr., both of the University of Virginia, have pointed out in such works as Reading: The Three Keys and The Knowledge Deficit, one of the keys to reading well is having the background knowledge that writers take for granted. Early in its existence, the Core Knowledge Foundation published a test of knowledge of the items on Hirsch’s famous “cultural literacy” list from his book Cultural Literacy. This test, which the foundation hasn’t used in a long, long time, is, nonetheless, an approximation of what a test of background knowledge for reading might look like, but it would only be useful if there were a curriculum that ensured that this knowledge was at least broached in K-12 classrooms. But I’m talking about a lot more than that. I’m talking about all the world knowledge (what psychometricians call crystalized, as opposed to fluid, knowledge) within the subject area of ELA. What elements do both parables and fables have in common? What obvious element differs in these two genres? What is a genre? What genre did Mary Shelley invent? What is her most famous work? How many feet are there in a hexameter line? How many lines in a sonnet? Which came first, the Enlightenment Era or the Romantic Era? What is Henry David Thoreau’s most famous work? Name two turn of the century Irish playwrights. What is a Bildungsroman? Which theory of criticism was championed by Brooks and Warren, Deconstruction, the New Criticism, the New Historicism, or Structuralism? What is another name for what DuBois called “the sorrow songs”? Who wrote “A Negro Speaks of Rivers”? What is the highpoint of interest or suspense in a fictional story called? Name four Beat poets. How does a metonymy differ from a synecdoche? Who did Samuel Johnson claim to be refuting when he famously kicked that rock? What character in the novel Frankenstein doesn’t have a name? In what language were the Upanishads written? To what fictional author are “Simple Simon” and “Who Killed Cock Robin” typically credited? Which name does not belong? Byron, Shelley, Pope, Keats, Coleridge, Wordsworth. What do we call an argument that starts with a hypothesis and then shows how it leads to a contradiction or absurdity? And so on. Content knowledge.
OK, gotta go one more time to putting my 2cts at bottom where I’ll get more margin space…
In 2002 No Child Left Behind Act (NCLB) mandated annual standardized testing in all 50 states. Prior to that, our state tested students in grades 3, 5 and 8, exactly what this article is proposing. High school students took mid-term and final exams. “Between 2009 and 2019, the number of teenagers facing persistent feelings of sadness and hopelessness increased 40 percent, according to historical data from the CDC. This figure increased another 20 percent when high school students were surveyed again in 2021, amid the pandemic”. One of the reasons given for this increase is “Teens are facing more academic pressure.” If students are experiencing burn out and depression from annual standardized testing, that’s another reason to end the annual standardized testing. The NCLB mandate needs to be revised. It has been in place over 20 years and has done little to help failing schools in impoverished neighborhoods.
Yup. The CDC just came out with a new study showing that too many adolescents, particularly girls, are in crisis mode. https://www.pbs.org/newshour/health/cdc-data-shows-u-s-teen-girls-in-crisis-with-unprecedented-rise-in-suicidal-behavior
AlwaysLearning– “It has been in place over 20 years and has done little to help failing schools in impoverished neighborhoods.”
Yup. In fact, nada, as far as I can determine. Not just for kids in impoverished neighborhoods, but for kids anywhere. NAEP, whatever its failings may be, has served as a corrective POV on achievements claimed via cut-score-manipulated state-stdzd scores for the last 20+ years. From its start in 1971, it showed small but steady incremental progress for all groups/ subgroups, until 2004 [full implementation of NCLB?], at which point the rate of increase started decreasing for a few years, then flattened by 2008. Over the last decade it has actually dipped in a few regions…
Love Dr. Berliner’s work. However, I will note that I posted here way back in 2017, I think it was, that public schools could save billions each year by ditching these invalid and pedagogically useless tests. That educational administrators at the federal, state, and local levels still, some of them, take these tests and their results at all seriously speaks volumes about the poor quality of their preparation for their jobs. So many morons working in these positions. The tests serve only the test makers. And they have led to a dramatic devolution in ELA and Mathematics pedagogy and curricula. It’s LONG past time to trash these.
The 1.7 billion doesn’t begin to reflect the actual cost of this ridiculous, invalid, pedagogically useless federally mandated standardized testing. It does not include costs for computers for kids to take the tests and practice tests on. It does not include the costs of test prep materials, print and online, that schools and districts purchase. It does not include parent outlays for test prep. It does not include the costs of textbooks and ancillary materials that have been rewritten and dumbed down to be test preppy. It does not include pay for school and district personnel to administer and proctor the tests. I suspect that that figure is off by a factor of ten.
And it certainly doesn’t include a) the opportunity costs for the education that students do not receive because resources and time are diverted to test-taking, practice testing, data conferences, and other such bs or b) the long-term consequences for the U.S. economy of the time and resources diverted into the testing black hole and away from actual instruction.
I think once, long ago, principals/ asst principals and supts of schools were people who moved up through the ranks from long years as teachers. It was true in my hisch day 60 yrs ago. Yes, the VP in charge of discipline was often a former athletic coach, but they never became principal.
I’ve kept in touch with trends anecdotally via younger sis who is an asst princ in the schdistrict we both attended in upstate NY—still a good hisch despite much larger % of poor kids these days. For at least the last 15-20 yrs [maybe longer], principals tend to be people who had a few yrs of teaching, then got MBA’s [et al admin certs]. Which makes sense, as our pubschsys has devolved into a data-focused enterprise, where schools are “graded”/ sanctioned according to multiple data.
That her school does rather well is due to its structure: that MBA-educated principal is propped up by 4 asst principals who all got there by being outstanding teachers with leadership experience. Among themselves they divvy up the 250 most at-risk students [of 1295; 20%], work with them and communicate with their families 24/7. And that’s just a side-gig: each has a specialty. My sis—background SpEd—works particularly well with delinquents, & is in charge of school safety. They also divvy up monitoring/ guiding teachers, as well as the gruntwork of handling scheduling snafus/ finding-assigning substitutes, et al.
The principals have usually been wafflers who don’t have your back, focused on PR and answering to mandates from state via supts [& of course the principals provide the fodder for the supt position]. So far in this anecdotal hischsys, asst principals never become principals: the latter are MBA types who wander from small schools to larger schools… And need I say thy are all men, while their ass princs are all women?
My experience has been the same, though I did have one female principal.
Here’s the other thing, when I first taught back in the 1980s, my principals (I was in three different schools) all consider the teachers the subject matter experts. Now, these morons think that they have the expertise to micromanage what people are teaching because they attended some workshop on data-driven instruction.
And have a copy of the puerile Gates/Coleman bullet list.
Well said. NCLB ushered in an era of devolution in public education and corporate influence.
Hi Bob, I remember your post about abolishing tests, and I have sympathy for that point of view. But too many folks seem to want tests of the type we commonly use, so I wanted to see if real data and the logic from the data could at least get them to test every third year…That not my endorsement of kinds of testing we developed in the USA.
Cordially
David
I am a HUGE fan of your work, Dr. Berliner!!! Thank you so much for the wisdom and knowledge and compassion for kids and teachers that you bring to these questions. And, as always, your piece was impeccably argued.
https://bobshepherdonline.wordpress.com/2020/03/19/why-we-need-to-end-high-stakes-standardized-testing-now/
The irony continues…
Haven’t persuaded the
“professional educators”
to STOP giving tests,
but continue to try to
persuade the “others”,
that their “pearls” are
nothing to clutch…
The lobbying over ESSA was intense, the AFT argued for testing every third year, a coalition of Civil Rights organizations lead by Wade Henderson opposed, he argued it was essential to have annual testing to keep the headlights on the discrepancy in test scores and the need for targeted additional funding, without support from a wide swath of Civil Rights organizations it is unlikely changes will be made, for example. would the NAACP and Ed Trust agree with you?
Testing results correlate to family income levels. More testing does nothing to change socioeconomics. The results generally stack up the same year after year. Perhaps making the child tax credit a permanent fixture might have an impact, but not enough politicians are willing to fight for it.
If Civil Rights organizations were filled with educators from underserved (impoverished) communities, they would agree. We’ve been shining those headlights on poor, minority children in grades 3 to 8 for over 20 years with absolutely no significant improvement beyond that which comes from familiarity/test prep and cut score manipulation.
Besides, we have better data than spurious standardized test score data in just two subjects. We have graduation rates, absentee rates, college admission rates, GPAs, AP and IB offerings and enrollments, and ongoing poverty rates these same communities. We don’t need no stinkin’ tests to tell us that generational poverty and dependence adversely affect the motivation needed for school success.
The Civil Rights organizations emphasize dramatic underfunding, implied bias, lack of teachers of color, absence of culturally relevant curriculum… the gap is significant… can Civil Rights organizations and anti- testing folk come together?
Not as long as Gates keeps pouring millions into buying support for standardized testing.
The civil rights groups used to vehemently oppose standardized testing as racist and classist. They were not wrong. The nature of morning the tests guarantees that there will always be achievement gaps, that the students with the most privilege will cluster on top and those with the least privilege will cluster at the bottom.
When Gates began dispensing millions of dollars to civil rights groups and told them that standardized testing is necessary, they became supporters of the instrument that harms them.
Peter– This has been the story with NCLB [now ESSA] since Ted Kennedy [backed by NAACP et al civil rights groups] shook hands with GWB over NCLB. My understanding is that Kennedy withdrew his handshake when Congress passed it as an unfunded mandate. Doesn’t do much good to highlight low-performing black schools if you’re not going to follow up with $$ for improvement. Wonder why Civil Rights groups don’t wake up to the 20+- yr reality that annual testing only hurts their kids’ learning & gets them zero additional fed support.
NCLB identified low-scoring schools, not to help them but to close them.
Worst education legislation in our history.
It continues under a new name with minor changes.
Civil Rights organizations are just as frustrated- wide racism, paucity of teachers of color, lack of culturally relevant curriculum…
The supreme irony of the testing madness is that the people who deemed it necessary to test everyone as often as possible didn’t take such tests in order to become smart enough to deem everybody else should take them. In my days in school–Michigan, Ohio public schools, Ohio State University–we didn’t have or take ACT’s, proficiencies, annual tests, etc. That was true in a number of states. The testing requirements were ordered for a population of people that led the world in technology and income–not to mention militarily and politically. Why did we let this happen? Sigh. Berlinger was one who warned us about what was happening. The question is, “What is to be done?” While blogging keeps ideas alive, as does Berlinger’s work, it will take action–or inaction, as in boycotts–to end the madness.
I’d love to ditch standardized testing entirely as I’ve never found it to be useful tool. As long as testing is considered accountability, I think we have little chance of eliminating it. It should, however, be no more than two mornings of drudgery every two or three years with no test prep or high stakes. The test should also be a standardized test that has been validated and normed. These state tests in which “proficiency” is considered the standard are pure bunk. Proficiency can mean whatever the powers that be want it to be. Proficiency can be a political tool. If they want lots of failures, set proficiency point high. If they want lots of students to pass, they can set the bar low. Proficiency is a subjective term. Deformers have created a lot of mischief based on “proficiency.”
Yes. Thinking as a negotiator and one who worked in arbitration, I’d say let’s propose getting rid of them–then your suggestions as to limitation is a good compromise.
So rarely do I meet people knowledgeable enough to understand that teachers and education products and programs cannot raise test scores. Those who don’t read the appropriate literature on the subject are doomed to make themselves and the students who toil under them suffer because of their gullibility. It’s one thing to have been fooled once, but to stay fooled for an entire career, never learning, an entire lifetime, shameful. And the people in the U.S. Department of Education, buying the same sales pitch as the tech industry was selling a quarter of a century, with all signs pointing to failure, never trying to fix the ESSA, they are worse than shameful. They are the lowest.
Don’t blame the U.S. Department of Education for the failure to fix NCLB/ESSA. You can blame the Secretary of Education because his or her voice would make a difference. But it’s Congress that writes and revises the law. Why are so many members of the Education Committees in the House and Senate are completely ignorant about the harm their laws do?
Thank you for that, Diane! I can’t tell you the countless # of commenters to ed articles & JQPublic voices on CSPAN who blame the Dept of Ed for over-testing, all unaware that they are merely implementing Congressional NCLB [2001] & ESSA [2015] laws.
Their own Congressional reps have put these policies in place, with the message that the nation’s pubschsystem is cheating its middle-class funders, needs roping in and forced to account for its “failing” performance or be sanctioned [$$ withdrawn, be taken over by state, be converted to charters].
Bipartisan pandering to the notion one can’t trust one’s democratic institutions! Fostering the concept that govt is “other,” not a reflection of locally-elected Bds of Ed, i.e. themselves!
Sickening that this has been a bipartisan movement—with the Dems chiming in cuz, neoliberalism/ privatization. The civil rights groups look to me like stooges caught in the middle of moneyed interests’ agendas—oblivious, shooting selves in the foot.
Is not Secretary Cardona the head of the Department of Education? If I thought I could work for the DoE without obeying the chain of command, working to end the federal testing mandate, I would have accepted the invitation to apply for the job. (Surely to be turned down for honest answers to interview questions.) And doesn’t Sec Cardona in some way answer to the president? Yes, Congress holds most of the blame, but the!bully pulpit is powerful, so the pres holds some of the blame. And the B&M Foundation, et al.
From my Los Angeles perspective, the federal government mandates annual, summative tests. The state mandates two interim tests a year. The district mandates the interim tests that require hours of hand scoring and also mandates three more tests that an unaffiliated, private company produced.
Congress and president, STOP WASTING MY TIME and everyone’s money! Gates, stop, well, everything, greedy, bizarre man.
RE: Duane/ Bob thread on testing up above… Let’s just dial it back to my question [far] above: can a state-stds-aligned assessment substitute for a teacher-designed assessment [or series of assessments throughout the year] on what he/ she taught that year in that classroom in that course?
This is going to depend on two factors: (a)the quality of the state stds for the course [can one actually teach to them (or are they too vague/ broad, age-inappropriate)], and is there time enough in schyr to teach them all. And (b) differences in curriculum/ pedagogy chosen to teach to the stds: best curricular approach is left to local Bd of Ed/ sch admin agreement—which includes some wiggle-room for teachers to tweak for their students– & best pedagogical methods for students in front of you is teachers’ domain.
IMHO, I don’t see how some state-stdzd exam, developed remotely from classroom, can hope to compete with a teacher-designed exam based on curriculum and pedagogy actually in place—as well as how many of those state stds could actually, reasonably covered in one schyear.
To go a step further– the idea that a state-stdzd exam could be superior in assessing how students on the ground respond to the actual curriculum/ pedagogy used [let alone to whether the # of state stds for the course could be covered in a schyr]– to a teacher-designed exam actually based on what they taught that year– says to me nothing more than that the state is out of touch, doesn’t trust the professionalism of its teachers, and has taken upon itself a task for which it has no expertise/ experience.
For contrast, let’s consider the old-timey NYS Regents exams. Every question was field-tested for 3 yrs, and there was a regular feedback-loop from a squadron of selected teachers back to the state dept of ed. And those exams did NOT substitute for teacher-designed final exams—they were an additional data point relating to whether you got credit toward a college-bound diploma [vs a regular diploma for those immediately entering the workforce—which no longer exists!]
There was a time before states thought that they had to micromanage education via bullet lists of so-called “standards,” and education in this country was a LOT better.
The so-called “standards” have always been bs.
Well said, well argued, Ginny.
Agreed. Competent teachers do not need the state telling them what to do. They know what’s best for their students. That system worked for at least seventy-five years until the wealthy got involved in undermining public education in order to privatize it.
“can a state-stds-aligned assessment substitute for a teacher-designed assessment [or series of assessments throughout the year] on what he/ she taught that year in that classroom in that course?”
NO!
Oh, one can substitute such a test but it doesn’t mean that it is valid or ethical.
When I read something by a so-called education “leader” or “researcher” who takes the results of the standardized state tests in ELA at all seriously, i conclude that there are three possibilities:
a. the person is a moron
b. the person is in the employ, directly or at some remove, of Billy Gates
c. both a and b
❤️❤️❤️❤️
I’m sure the folks in the standardized test industry are celebrating and promoting these excellent ideas.
lol. Same for all who feed at the Gates trough.
Great piece on “Storytelling”, Bob.
My sense of metacognition has increased dramatically since retirement. As a result I’ve become very aware of how often I create stories during the course of the 24 hour day.
“Yep…there I go, again. What is it that keeps making me do this, anyway?”
Your essay was extremely informative and helpful. Thank you.
Thanks a million, Gita!!!
I assume, gitapik, that you are being ironic
Best, ofc, to do away with the federally mandated state standardized tests altogether. A far better alternative, given the idiocy of the tests and their pedagogical uselessness and their cost in teacher, student, and administrator time, in money, in opportunity, teacher and student frustration, lost instructional time, parental grief, and so on.
When will the teachers’ unions take a stand in the streets to end the federal testing mandate? Until they do, they are complicit in a) child abuse and b) the devolution of U.S. curricula and pedagogy to make it test preppy and to base it on the puerile, almost content free Gates/Coleman “standards” foisted on the country with ZERO vetting.
Hi Bob,
I have some good news. I have been working a bit with the aide to congressman Jamaal Bowman (an ex-teacher). He has a bill that may get to the floor at the end of the month to reconsider all the federal mandates, particularly the testing craziness. We’ll all need to support him when/if it gets to the floor. So lets all watch for it and make sure our own congressperson gets our input if the bill gets a hearing.
David
Thanks, David. That is great news indeed.
Yes. Definitely ironic.
While I respect and agree with your statements on a philosophical level, I can’t go along with the “complicity” aspect of your post, in a realistic sense.
I was in the trenches when Bloomberg was bulldozing the NYCDOE. Teachers were sent to The Rubber Room (airless, windowless basement rooms in schools located far from that teacher’s home address) for as little as raising a voice in protest of the unrealistic changes and expectations. Some were fired…and the many who weren’t were made to want to resign.
It’s a big deal; risking the loss of your livelihood. Especially when the deck is so stacked against you. Gates and others used the financial difficulties of the Great Recession to promote their agendas. It’s one thing to battle the behemoth. Even worse when your elected officials (governmental and union) aren’t a lock to back you up.
I’m not talking, gita, about rank and file teachers. I’m talking about the national union leadership. This has to be a nationwide thing, LED by the leaders of the major unions.
Well said, Bob. And, aye: therein lies the rub.
Our national union leadership seems to have had an eye on the public trend, the past 2+ decades. Think of Gates as keynote speaker at the 2010 AFT convention. And more.
Then there are the locals. I’ve had issues with our leadership, here in NYC. I know it’s been tough…but that’s when we’re supposed to try harder in my book.
But the overall picture seems to be pretty good for union growth and power. Hoping it continues.
Instead of taking Gates’s money, those leaders could call for the ending of the federal mandate and, say, a nationwide walkout to make the point.
We have allowed this utter bs to go on far, far too long.
It’s like a foreign occupation of our public schools, where Seattle plays the role that Berlin did in WWII.
Words fail with regard to how disastrous the Gates/Coleman-inspired testing madness has been for our country (yes, I am quite aware that the origins of this madness predate them; I know the history). The cost has been staggering. And Bill Gates, clueless freaking Bill Gates, has no idea what he has done. None whatsoever. Talk about a freaking slow learner. I doubt seriously that he has ever compared a U.S. high-school literature text before and after his idiotic initiative to see how much these have devolved.
Clueless. Totally clueless.
Clueless and surrounded by suckling sycophants.
But as a K-12 educational publishing exec, I saw this devolution up close and personal. Frightful. Horrific. Brain dead.
But yes, Gita. Under the occupation, a LOT of teachers learned quickly to keep their heads down. The best of them shut their doors and continued to teach English or Math DESPITE the testing and the Coring of U.S. pedagogy and curricula. That’s what I did.
“But yes, Gita. Under the occupation, a LOT of teachers learned quickly to keep their heads down. The best of them shut their doors and continued to teach English or Math DESPITE the testing and the Coring of U.S. pedagogy and curricula. That’s what I did.”
Me, too. And I got caught. My AP pulled me into her office (shades of grade school in the ’50s…it was glorious) and told me, “If I see you using those books again; I’ll personally burn them”.
🤜❤️🤛
Yikes!!!!!!!!!
I was delivered new copies of the Pear$son My Per$pectives series. I did a review of them, wrote an exhaustive and utterly damning critique of them and sent this to my fellow English teachers and principal and left them in boxes in my classroom for the rest of the year, taking them out only if they happened to have a selection I wanted to use. Garbage. Absolute garbage.
And good for you!
It’s been a great pleasure discussing this with you, gp.
You too, Bob!
the Coring, the hollowing out, of U.S. ELA curricula and pedagogy
This has gone on for so long that the younger English teachers have never known anything else. They think that this farce is normal.
And it is normal in the sense of “the norm” now.
“This has gone on for so long that the younger English teachers have never known anything else. They think that this farce is normal.”
I know. I saw it towards the end of my tenure and I see it now, when I sub as a retiree.
When I talk with younger teachers and bring up the concept that it wasn’t always this way; it becomes a “back in the day” conversation…which is pathetic.
The plan seems to have been to outlast us. So far they’ve succeeded. Hopefully this new tilt towards unionization will create some change.