David Berliner and Gene Glass are leaders of the American education research community. Their books are required reading in the field. They shared with me their thoughts about the value of annual testing in 2021. I would add only one point: if Trump is voted out in November, Jim Blew and Betsy DeVos will have no role in deciding whether to demand or require the annual standardized testing regime in the spring of 2021. New people who are, hopefully, wiser and more attuned to the failure of standardized testing over 20 years, will take their place.
Glass and Berliner write:
Why Bother Testing in 2021?
Gene V Glass
David C. Berliner
At a recent Education Writers Association seminar, Jim Blew, an assistant to Betsy DeVos at the Department of Education, opined that the Department is inclined not to grant waivers to states seeking exemptions from the federally mandated annual standardized achievement testing. States like Michigan, Georgia, and South Carolina were seeking a one year moratorium. Blew insisted that “even during a pandemic [tests] serve as an important tool in our education system.” He said that the Department’s “instinct” was to grant no waivers. What system he was referring to and important to whom are two questions we seek to unravel here.
Without question, the “system” of the U.S. Department of Education has a huge stake in enforcing annual achievement testing. It’s not just that the Department’s relationship is at stake with Pearson Education, the U.K. corporation that is the major contractor for state testing, with annual revenues of nearly $5 billion. The Department’s image as a “get tough” defender of high standards is also at stake. Pandemic be damned! We can’t let those weak kneed blue states get away with covering up the incompetence of those teacher unions.
To whom are the results of these annual testings important? Governors? District superintendents? Teachers?
How the governors feel about the test results depends entirely on where they stand on the political spectrum. Blue state governors praise the findings when they are above the national average, and they call for increased funding when they are below. Red state governors, whose state’s scores are generally below average, insist that the results are a clear call for vouchers and more charter schools – in a word, choice. District administrators and teachers live in fear that they will be blamed for bad scores; and they will.
Fortunately, all the drama and politicking about the annual testing is utterly unnecessary. Last year’s district or even schoolhouse average almost perfectly predicts this year’s average. Give us the average Reading score for Grade Three for any medium or larger size district for the last year and we’ll give you the average for this year within a point or two. So at the very least, testing every year is a waste of time and money – money that might ultimately help cover the salary of executives like John Fallon, Pearson Education CEO, whose total compensation in 2017 was more than $4 million.
But we wouldn’t even need to bother looking up a district’s last year’s test scores to know where their achievement scores are this year. We can accurately predict those scores from data that cost nothing. It is well known and has been for many years – just Google “Karl R. White” 1982 – that a school’s average socio-economic status (SES) is an accurate predictor of its achievement test average. “Accurate” here means a correlation exceeding .80. Even though a school’s racial composition overlaps considerably with the average wealth of the families it serves, adding Race to the prediction equation will improve the prediction of test performance. Together, SES and Race tell us much about what is actually going on in the school lives of children: the years of experience of their teachers; the quality of the teaching materials and equipment; even the condition of the building they attend.
Don’t believe it? Think about this. In a recent year the free and reduced lunch rate (FRL) at the 42 largest high schools in Nebraska was correlated with the school’s average score in Reading, Math, and Science on the Nebraska State Assessments. The correlations obtained were FRL & Reading r = -.93, FRL & Science r = -.94, and FRL & Math r = -.92. Correlation coeficients don’t get higher than 1.00.
If you can know the schools’ test scores from their poverty rate, why give the test?
In fact, Chris Tienken answered that very question in New Jersey. With data on household income, % single parent households, and parent education level in each township, he predicted a township’s rates of scoring “proficient” on the New Jersy state assessment. In Maple Shade Township, 48.71% of the students were predicted to be proficient in Language Arts; the actual proficiency rate was 48.70%. In Mount Arlington township, 61.4% were predicted proficient; 61.5% were actually proficient. And so it went. Demographics may not be destiny for individuuals, but when you want a reliable, quick, inexpensive estimate of how a school, township, or district is doing in terms of their achievement scores on a standardized test of acheievement, demographics really are destiny, until governments at many levels get serious about addressing the inequities holding back poor and minority schools!
There is one more point to consider here: a school can more easily “fake” its achievement scores than it can fake its SES and racial composition. Test scores can be artificially raised by paying a test prep company, or giving just a tiny bit more time on the test, looking the other way as students whip out their cell phones during the test, by looking at the test before hand and sharing some “ideas” with students about how they might do better on the tests, or examining the tests after they are given and changing an answer or two here and there. These are not hypothetical examples; they go on all the time.
However, don’t the principals and superintendents need the test data to determine which teachers are teaching well and which ones ought to be fired? That seems logical but it doesn’t work. Our colleague Audrey Amrein Beardsley and her students have addressed this issue in detail on the blog VAMboozled. In just one study, a Houston teacher was compared to other teachers in other schools sixteen different times over four years. Her students’ test scores indicated that she was better than the other teachers 8 times and worse than the others 8 times. So, do achievement tests tell us whether we have identified a great teacher, or a bad teacher? Or do the tests merely reveal who was in that teacher’s class that particualr year? Again, the makeup of the class – demographics like social class, ethnicity, and native language – are powerful determiners of test scores.
But wait. Don’t the teachers need the state standardized test results to know how well their students are learning, what they know and what is still to be learned? Not at all. By Christmas, but certainly by springtime when most of the standardized tests are given, teachers can accurately tell you how their students will rank on those tests. Just ask them! And furthermore, they almost never get the information about their students’ acheievement until the fall following the year they had those students in class making the information value of the tests nil!
In a pilot study by our former ASU student Annapurna Ganesh, a dozen 2nd and 3rd grade teachers ranked their children in terms of their likely scores on their upcoming Arizona state tests. Correlations were uniformly high – as high in one class as +.96! In a follow up study, with a larger sample, here are the correlations found for 8 of the third-grade teachers who predicted the ranking of their students on that year’s state of Arizona standardized tests:
In this third grade sample, the lowest rank order coefficient between a teacher’s ranking of the students and the student’s ranking on the state Math or Reading test was +.72! Berliner took these results to the Arizona Department of Education, informing them that they could get the information they wanted about how children are doing in about 10 minutes and for no money! He was told that he was “lying,” and shown out of the office. The abuse must go on. Contracts must be honored.
Predicting rank can’t tell you the national percentile of this child or that, but that information is irrelevant to teachers anyway. Teachers usually know which child is struggling, which is soaring, and what both of them need. That is really the information that they need!
Thus far as we argue against the desire our federal Department of Education to reinstitute achievement testing in each state, we neglected to mention a test’s most important characteristic—its validity. We mention here, briefly, just one type of validity, content validity. To have content validity students in each state have to be exposed to/taught the curriculum for which the test is appropriate. The US Department of Education seems not to have noticed that since March 2020 public schooling has been in a bit of an upheaval! The chances that each district, in each state, has provided equal access to the curriculm on which a states’ test is based, is fraught under normal circumstances. In a pandemic it is a remarkably stupid assumption! We assert that no state achievement test will be content valid if given in the 2020-2021 school year. Furthermore, those who help in administering and analyzing such tests are likely in violation of the testing standards of the American Psycholgical Association, the American Educational Research Association, and the National Council on Measurement in Education. In addition to our other concerns with state standardized tests, there is no defensible use of an invalid test. Period.
We are not opposed to all testing, just to stupid testing. The National Assessment Governing Board voted 12 to 10 in favor of administering NAEP in 2021. There is some sense to doing so. NAEP tests fewer than 1 in 1,000 students in grades 4, 8, and 12. As a valid longitudinal measure, the results could tell us the extent of the devastation of the Corona virus.
We end this essay with some good news. The DeVos Department of Education position on Spring 2021 testing is likely to be utterly irrelevant. She and assistant Blew are likely to be watching the operation of the Department of Education from the sidelines after January 21, 2021. We can only hope that members of a new admistration read this and understand that some of the desperately needed money for American public schools can come from the huge federal budget for standardized testing. Because in seeking the answer to the question “Why bother testing in 2021?” we have necessarily confronted the more important question: “Why ever bother to administer these mandated tests?”
We hasten to add that we are not alone in this opinion. Among measurement experts competent to opine on such things, our colleagues at the National Education Policy Center likewise question the wisdom of a 2021 federal government mandated testing.
Diane Ravitch's blog 
People can post the data all they want and we can talk about it on this blog every day, all day, but until parents accept the truth and are willing to change the narrative, nothing will change. High test scores equate to social status and our standings within our community….. and socio/economic status is what matters to Americans. We live in a country that glorifies Me and Mine and the lifestyles of the rich and famous and it doesn’t matter how it is obtained or who got squashed in the process. Democracy is about “We , competition is about “Me”…..The US has become a “Me and Mine” Libertarian nightmare.
Eugenics as reason for racial/cultural/elitist standing
When I first started teaching elementary ELLs in New York State. standardized tests were only administered in grades 3, 6 and 8, and nobody paid attention to or taught to the tests, There were no high stakes attached to them. We were told that the tests were like a litmus test. Scores on standardized tests were reported in percentiles. There was no such thing as “proficiency” levels. ELLs took an annual English assessment that could be administered in class. Very little time was wasted on standardized testing.
By the time I left teaching more than thirty years later, testing had taken on a life of its own. Accountability was king! Students and teachers were stressed from all the tests which consumed too many spring mornings. My ELLs had to take all the tests the other students took and also a long standardized NYSESLAT for English language learners. When I calculated the number of instructional mornings that were gobbled up by testing, I found that my students lost 28 prime time instructional mornings wasted on standardized testing. Students are losing too much instructional time due to our testing obsession. The results that arrive at the very end of the school year do nothing to inform or improve instruction.
For this school year our district will focus on safety first, social emotional second and academics come last – on paper, in theory. But I have a feeling we will continue to follow the assess and “accountability” model in practice.
I just read this in the Educator’s Room emailed newsletter. I thought it captured our current state of teaching:
https://theeducatorsroom.com/dont-be-fooled-the-fall-will-be-difficult-but-teachers-were-demoralized-long-before-covid-19/?utm_source=newsletter&utm_medium=email&utm_campaign=since_youre_in_quarantine_read_these_articles_by_teachers&utm_term=2020-08-14
How do ed reformers explain that they lobby for and receive exemptions to standardized testing for the publicly-funded private schools they prefer but also lobby for mandated standardized tests in public schools?
“The law specifies that alternative exams will be determined by ODE, and it does not appear to limit the department to picking just one alternative. That means it’s possible that private schools could have a list of options from which to choose, much like public schools do with the third grade reading guarantee. The upshot of all this is that, once the budget goes into effect, voucher students in Ohio will no longer be required to take the same state exams that the vast majority of their taxpayer-funded peers do.
This change wasn’t widely debated—at least not outside the closed doors of conference committee—but it was likely made in response to two common complaints by voucher advocates. The first is that more private schools would be willing to accept voucher students if they weren’t subject to state testing mandates. Private schools are zealous in protecting their ability to choose their own curriculum, materials, and assessments, and consider administering the state test—even to a small number of students—an infringement on their autonomy.”
Jim Blew spends most of his work day bashing public schools and promoting charters and private schools. Maybe someone can ask him why students at publicly-funded private schools aren’t forced to spend weeks taking standardized tests but public school students are. Other than the fact that there’s an ideological preference in the ed reform “movement” for private schools and private school students, I mean.
Why do public school students and families always get the short end of the stick in ed reform engineered schemes? Why should public school students and families put up with thousands of publicly-paid employees who don’t do any practical, positive work for their schools?
“Private schools are zealous in protecting their ability to choose their own curriculum, materials, and assessments, and consider administering the state test—even to a small number of students—an infringement on their autonomy.”
I case anyone had any doubt that the ed reform “movement” prefers private schools and private school students over public schools and public school students, their blatant hypocrisy on standardized testing makes it pretty clear.
They exempt their own schools from the incessant testing they mandate in our schools.
Apparently “accountability!” for publicly-funded schools doesn’t extend to the private schools that meet ed reform’s ideological preferences.
I don’t mind that they prefer private schools over public schools. That’s an ideological preference and they can hold it. I mind that they present themselves as “agnostics” to the public and they are not. That’s not fair to public school students and families who get screwed in each and every ed reform scheme and NEVER come out well when ed reformers are in charge.
We should KNOW if we’re hiring a bunch of public employees who don’t support our schools and students. They owe us that. They don’t return any value to public school students. That’s not acceptable.
The point of the NCLB testing regime and its ESSA follow up was not preventing poor achievement or Inequity. We knew then about the correlation with race and ses. We knew about inequitable funding. The purpose was and is orchestrating the illusion of evidence of failure in order to promote what Americans didn’t and don’t want, privatization and profitizing from education.
So TRUE, Arthur! 👍🏽👍🏽👍🏽
All this testing is just another money boondoggle for the testing companies who have lobbyists.
Get rid of lobbyists. They cost money and most of them have no clue about the value of a public school education.
Public Schools and Public School Teachers are two of our Nation’s treasures.
Jim Blew also has the (required) ed reform work history:
“Prior to joining the Department, Blew advocated for education reform across the country. His roles included serving as director of the 50CAN affiliate Student Success California, national president of StudentsFirst, and national director of the Alliance for School Choice and its predecessor, the American Education Reform Council.
Blew also helped guide the Walton Family Foundation’s K-12 reform investments for nearly a decade”
Echo chamber. They ALL come out of the same anti-public school lobbying groups, think tanks and organizations. They all sound exactly the same, down to words and phrases, because the echo chamber is rigidly enforced and no one from outside it is ever hired.
Our schools do poorly under ed reform leadership because they all come out of orgs that spend all their time lobbying to expand vouchers and charters and cut public schools. No other outcome was possible. The results are baked in at the hiring process and there’s NO dissent.
The public should ask themselves how they ended up with an entire elite education policy apparatus that is OPPOSED to the schools 90% of students and families use, and look at how that’s working out for public school students. Not well!
“Andrew Ujifusa
For every $1,000 cut from a school district’s budget, overall test scores fell, and academic gaps between white students and Black and Latino students grew by 6 percentage points, a new study of the Great Recession from
KiraboJackson and others says.”
That’s what ed reform delivered to public school students in the Great Recession. That’s the result of the thousands of full time “education advocates” who are employed in “the movement”. Public school students bore the brunt of the 2009 Wall Street crash.
Public school students will now bear the brunt of the pandemic.
Are ed reformers just lousy advocates or do they not really work on behalf of public school students? Why do public school students seem to fare so consistently poorly when these folks are in charge?
The window dressing on the test and punishment regime of the last several decades was shining a harsh light on the “achievement gap” and then “holding educators accountable.” Because the implied laziness and incompetence was never the problem, nothing changed. Well, wait, it did. It distorted teaching, so that test performance rather than education for life, work, and citizens became the goal. It succeeded in undermining public investment in public education and democratic governance and gave rise to a new industry, charter schools. In other words, it worked as planned, but only up to a point. Americans still value public education and trust its teachers. The job is now what it always was, ensure all families have what they need to live a decent life and establish equitable funding for all schools.
But think of all the unemployed edugeeks if we banish their pseudo-intellectual practice for a time.
Thanks to Gene Glass and David Berliner for again stating what should be well known about the national testing regime. Also note that this regime of testing, and anxiety about results, is almost exclusively about test scores in Math, ELA, and now and then Science.
These same preoccupations can be seen in the whole enterprise of educational research, including studies that recycle data from the absurd Gates-funded “Measures of Effective Teaching” study.
All of us who work in the arts and humanities and all of our students have been placed on the sidelines by the drive to aggrandize the importance of test scores in Math, ELA, (sans literature), with nods to science.
I just had an epiphany – the push to reopen school buildings is also an effort to keep the Ed testing corporate beast alive.
How do you test kids if they’re not in school buildings?
If kids aren’t tested, testing corporations lose money.
Pretty cynical.
But probably not far from the truth.
Ugh.
[…] not use language in support of their products that can match what their critics employ: “a waste of time and money”, “racist“, “corrupt, stupid, punitive, and just […]
[…] from teachers’ ignorance about testing should work, a prominent professor or two will claim that “teachers can accurately tell you how their students will rank on those (standardized) tests.” Therefore, those teachers must know something about assessment and even testing. […]