A few years bacon, the story of the “Mississippi Miracle” in reading was all the rage. The increase in scores of fourth grade students on NAEP scores was hailed as miraculous, a testament to the dramatic power of the “science of reading.” New York Times’ columnist Nicholas Kristof wrote a column praising Mississippi for raising the test scores of its fourth graders without spending any more money. Anyone could do it!
I was critical of Kristof’s enthusiasm and pointed out that the scores of fourth graders soared but the reading scores of eighth graders did not. The scores of the older students were among the lowest in the nation. What kind of “miracle” dissolves as students get older?
Thomas Ultican reviews the “Mississippi Miracle” and also finds it to be hype. But he sees it as good reason to kill NAEP, which Trump is now doing.
I don’t often disagree with Tom, who is a relentless researcher of scams and hoaxes perpetrated by the critics of public schools.
I oppose the misuse of high-stakes standardized tests to hold teachers, students, and schools “accountable,” because the tests are loaded with errors and inevitably reflect family income and family education, not the ability of students or teachers. I have written about the inherent flaw of standardized tests in my last three books.
What I like about NAEP is that it is a no-stakes test. It too reflects family income and family education, like all standardized tests. But no one is punished or rewarded for their test scores.
NAEP shows trends by states, cities, gender, race, ethnicity, special ed status, income, etc.
It is NAEP that reveals the lie behind the “Mississippi Miracle.” NAEP shows that fourth graders made dramatic progress and minimal sleuthing demonstrates that the lowest performing students were held back in third grade, excluded from the testing pool.
It’s NAEP that reveals that eighth graders placed 43rd of 50 states. The Miracle didn’t persist.
I think NAEP should remain and the federal mandate for testing every child every year in every school should be abandoned.
Diane Ravitch's blog 
NAEP suffers all the same invalidities as shown by Wilson and others as all other standardized tests. Yes, it has stakes. . . stakes that go right through the heart of what the teaching and learning process is. Usage of any of the results is false, invalid, risible and ludicrous. . . a waste of time, effort and monies.
LikeLike
No one knows which students or schools were tested. No student takes the entire test. There are no consequences for any student, teacher or school. Since no one knows what is tested, no test prep is needed. That’s about as low stakes as it gets.
LikeLike
Has DOGE killed NAEP?
LikeLike
In theory, DOGE did not kill NAEP. In reality, everyone who administered NAEP, awarded contracts, reviewed assessments and monitored quality–was fired.
LikeLike
It appears to be low stakes but since the whole process is invalid the stakes are raised in that interpreting invalid data is nothing more than and exercise in mental masturbation. Decisions are made from that data. There are stakes.
LikeLike
Being able to identify the words in a paragraph and being able to comprehend the meaning of the paragraph are two different things. Readers of this post could all probably read outloud an advanced physics textbook, but could everyone comprehend the material?
LikeLiked by 1 person
I tend to think that standards are a mirage, so standardized testing in any form is problematic. Still, I am also acutely aware that test scores were a political motivation for some schools I know personally to put funds and teachers in places that I knew helped students. The reality is that administrators have to choose between the reading program and the weight room. Almost no rural school I know of has a choir program in Tennessee. So what are we going to do? Testing has all but stamped out the arts in some schools, but other areas at least get some help.
maybe we should go back to the Napoleonic idea of the Normale, the school which was at a college level that established norms for learning. Teachers had to go to these schools, and there they learned what well-educated math students looked like. With a school full of teachers who can recognize learning, all you have to do is ask them to report on the learning of their students. For the most part, well trained teachers give a conservative estimate of their students’ progress that is of greater use than a test.
LikeLiked by 1 person
Roy,
Teachers know their students far better than standardized tests do. They know what they taught. They should write their own tests.
LikeLiked by 2 people
I could generally predict within 5 points how my students would perform on a standardized test. In class assessments are a far better approach and are far more relevant than standardized tests that mostly a waste of time and money, IMO.
LikeLiked by 1 person
Diane you an I disagree on this. You say “What I like about NAEP is that it is a no-stakes test.” I do not think it is no stakes. Every time NAEP data is released is accompanies by demagoguery and subterfuge. Even NAEP has a bit of a tendency to narrow curriculum.
I do not mind NAEP testing but the misuse and lying about the results seriously diminishes its value.
LikeLiked by 2 people
Tom,
I understand. The biggest NAEP reporting problem is the word “proficient.” Most journalists and the public think that almost everyone should be proficient. They demagogue the results by saying “most students are not proficient.” Proficient is not grade level. Proficient represents A or A- performance. If 90% of kids scored proficient, critics would complain about grade inflation.
I have urged the governing board of NAEP to use a different word, like “outstanding” or “mastery.” Something that signifies what proficient really is.
Frankly we would be better off if the achievement levels were eliminated and results were reported by scale scores.
Achievement levels were adopted in 1992, but Congress gave them only conditional approval until they had been thoroughly tested.
Fact is, the achievement levels are arbitrary and subjective, based on the judgments of non-experts. Every scholarly review has criticized them as fundamentally flawed.
LikeLiked by 1 person
Mississippi offers an example of understanding why NAEP must not be done away with. NAEP scores will always capture in them the behavior of the systems that produced the scores, no matter any foibles inherent in the NAEP testing process itself; perfection does not exist.
Mississippi NAEP Grade 4 Reading mean scale scores with respect to “All students” have continually increased since 1992. Scores have gone up nine times and down six times over the 16 NAEP administrations spanning 1992 through 2024. The continually increasing behavior of the scores has been stable hence predictable within a range. The Mississippi reading program that has been in effect since 2013 has made no detectable difference in the continually increasing behavior of the scores. However, the biggest increase ever showed up in 2015 and may have been an effect of that 2013 reading program. Even if that’s case, that biggest ever increase is not detectably different from any other score increases that have occurred. Biggest or highest or lowest ranked doesn’t necessarily mean different.
In contrast to Mississippi Grade 4 Reading, NAEP scores for Mississippi Grade 8 Reading have been flat since 1998. And their behavior has been stable hence predictable within a range since 1998. Only random variation is evident, with seven ups and six downs, so roughly an increase 50 percent of the time and a decrease about 50 percent of the time, as to be expected with random variation. Here, too, the Mississippi reading program has made no detectable difference.
Conclusion: No rational basis exists for claiming a “Mississippi Miracle” with respect to either or both that state’s grade 4 reading and grade 8 reading as assessed by NAEP.
So, some questions might be: Why have Mississippi grade 4 reading scores on NAEP continually increased since 1992 and are likely to increase more so in at least the near future? Why have Mississippi grade 8 reading scores of NAEP been flat since 1998 and are likely to remain flat in at least the near future? Why has the Mississippi reading program instituted in 2013 been ineffective?
Such questions pose opportunities to lean to improve systems and not simply react to what happened and argue over comparisons of results, when comparisons of results more often than not are meaningless. Again, Mississippi offers an example of understanding why NAEP must not be done away with. NAEP scores will always capture in them the behavior of the systems that produced the scores, no matter any foibles inherent in the NAEP testing process.
LikeLike
Thanks, Ed. My hypothesis about the “miraculous” rise in 4th grade reading scores in Mississippi is that the number of low-scoring 3rd graders retained (not promoted to 4th grade) increased. This artificially inflated 4th grade scores. Did the state advance them to join their age-group by 8th grade? I don’t knows.
What I do know is that a big improvement in 4th grade that disappears in 8th grade is no miracle.
LikeLike
A few years ago, Tennessee passed its own 3rd grade retention law based on “adequate growth” on state tests. I said at the time that it would eventually serve to inflate TN’s 4th grade NAEP scores.
LikeLike
Florida led the way with third grade retention, fourth grade miracle, flat scores in eighth grade. Fools the media every time
LikeLiked by 1 person
Diane, neither NAEP nor any other agency’s so-called “achievement levels” have ever made any intuitive sense to me in any kind of way that tells a holistic story. So, some months ago I went seeking to discover a way to tease such a story out of any given set of achievement level percentages taken all together. After much trial and error, I hit upon a method that calculates a 0-100 percent quality measure of fitness for the presumptive purpose of teaching and learning, or what I call “Adequacy.” Because the method uses all achievement level percentages, it cannot be about any students at any one achievement level—NAEP “at Proficient,” for example. It can only be a quality measure of the educational environment or system students are in. Which is another reason NAEP is valuable—that is, using results as system assessments rather than student assessments.
Although continually increasing, Mississippi grade 4 reading has averaged 24.6% Adequacy over 1992-2024, having gone from 18.9% Adequacy in 1992 to 34.5% Adequacy in 2024.
Mississippi grade 8 reading has averaged 28.5% Adequacy. For any given NAEP assessment, Adequacy anywhere between 25.3% and 31.7% is normal and period-to-period comparisons meaningless.
So, Adequacy tells me an intuitively holistic story about a system’s fitness for the presumptive purpose of teaching and learning.
LikeLike
Ed, see my comments below about invalidity and unethical usage of NAEP results.
LikeLike
Duane, I saw your comments on don’t dispute the reality you speak. Still, on the flipside, another reality is that NAEP results may be used in valid and ethical ways. Like every measuring instrument, NAEP testing isn’t perfect, so NAEP results aren’t perfect, as I said. Nonetheless, NAEP results will always have captured in them something about the behavior and quality of the educational systems to which the NAEP testing process is applied. Thus, a valid and ethical quality improvement opportunity is to extract and use knowledge locked up NAEP results about the educational systems that produce the results. It’s not so much about testing students—the prevailing paradigm–as it is about a different paradigm of “hearing” NAEP results represent the Voice of Students speaking about the quality of the educational systems within which teaching and learning occurs.
LikeLike
Nope!
LikeLike
How Not to Teach a Child to Read and How to Do It Properly
Two more videos coming in this series. –Bob S.
LikeLike
First video in this series on Reading (intro):
Resources for Teachers: How Not to Teach a Child to Read and How to Do It Properly, Part 1
LikeLike
If NAEP was actually randomly selecting schools and testing, fine. But my school has had to test 8th graders like clockwork every 2 years for the past 10-12 years. I don’t believe that is “random,” and it does eat up a lot of resources that my school doesn’t have.
LikeLike
TOW,
I don’t know why your school has been repeatedly chosen. As a member of the NAEP board, we were told by the contractors that students and schools are randomized. No individual student takes the entire test. Even if your school was tested repeatedly, it is never identified, nor are any students. The test has no consequences for your school or the students. That’s no-stakes.
LikeLike
Mississippi miracle = Texas miracle*
*see Rod Paige
LikeLike
Using the results of a test for anything other than what the test purport to assess is invalid and unethical. NAEP supposedly assesses reading and math capabilities. It is not designed to evaluate teaching and learning processes. Hence, using NAEP results as is done is the definition of invalid and unethical usage.
LikeLike
Actually, Mississippi is starting to show improvement in Grade 8 NAEP Reading compared to other states. But you have to break things out in order to see it. If you only look at overall average scores, you will fall into the trap discussed on Page 32 in the 2009 NAEP Science Report Card.
Data extracted from the NAEP Data Explorer shows in 2013 when its reform began, Mississippi’s White students ranked 49th compared to White students in the other states and scored statistically significantly lower than the national public school average. By 2024, Mississippi moved up to 31st place and was in a statistical tie with the national public school average for White students’ scores.
The situation was even more dramatic for Black students. In 2013, Mississippi’s Black students ranked 41st out of the 42 states that got scores for Black students. By 2024, Mississippi’s Black students ranked 14th out of the 38 states that had scores reported for Black students. As with the Whites, Mississippi’s Blacks scored statistically significantly lower than the national public school average in 2013 and tied it in 2024.
The belief that Mississippi’s scores are only better due to 3rd grade retention has received more scrutiny and doesn’t hold up, either. One example is a discussion by Todd Collins in his 2023 blog, “Mississippi didn’t cheat. Its reading gains are real.”
Of note, Collins was perhaps the first to raise the issue back in 2019 when he at that time believed 3rd Grade retention was a major player in Mississippi’s rise in NAEP Grade 4 Reading rankings.
LikeLike
So agree, Diane. It is foolish to contemplate abolishing the Nation’s Report Card simply because its scores are the result of standardized tests. They are the only guardrail/ reality check we have against NCLB/ ESSA annual state-stdzd tests, which cannot be compared apples to apples (state to state), and often enough can’t even measure progress within an individual state from one year to another, because they often change testing companies. Furthermore, NAEP give us a retrospective on that one, admittedly slim measure [standardized biennial tests with a history of 54 years of reporting].
LikeLike
If we had teachers’ unions worthy of the name, they would long ago have called for nationwide walkouts/strikes to end the federal testing mandate. Until they do that, they are complicit in child abuse.
LikeLike
Come on union leaders. Be heroes. Do something.
LikeLike
This is so silly.
A) Those kids who are held back in third grade still have to take the test. They take it one year later, of course, but they still have to take the test.
B) The 8th graders aren’t doing as well because the program was only four years old when these 8th graders were in kindergarten! Wait four years and you will see these current gains applied to eighth graders.
I refuse to believe someone could possibly be shortsighted enough to miss both of these very obvious points.
LikeLike