Paul Thomas was a classroom teacher for many years in South Carolina. He decided to become a professor of education, and eventually joined the faculty at Furman University, first class liberal arts institution in South Carolina.

He writes here about the improbability of miracles. I disagree with Paul Thomas on one point: Miracles are not only unlikely or improbable. There are NO miracles in education. My friend Mike Klonsky of Chicago said to me years ago. “If you are looking for a miracle, go to church, not to school.”

In all my years, I have found no reason to doubt this wisdom.

Paul Thomas writes:

My entire career in education, begun in the fall of 1984, has been during the accountability era of education that is primarily characterized by one reality—perpetual reform.

The template has been mind-numbingly predictable, a non-stop cycle of crisis>reform>crisis>reform, etc.

Another constant of that cycle is that the crisis-of-the-moment has almost always been overblown or nonexistent, leading to reforms that fall short of the promised outcomes. Reforms, ironically, just lead to another crisis.

But one of the most powerful and damning elements in the crisis/reform cycle has been the education miracle. [1]

Two problems exist with basing education reform on education miracles. First, and overwhelmingly, education miracles are almost always debunked as misinformation, misunderstanding of data, or outright fraud. Research has shown that statistically education miracles are so incredibly rare that they essentially do not exist.

Second, even when an education miracle is valid, it is by definition an outlier, and thus, the policies and practices of how the miracle occurred are likely not scalable and certainly should not be used as a template for universal reform.

Those core problems with education miracles have prompted the attention of Howard Wainer, Irina Grabovsky and Daniel H. Robinson, who have analyzed the reading reform miracle claims linked to Mississippi:

In 1748, famed Scot David Hume defined nature. He elaborated such a law as “a regularity of past experience projected by the mind to future cases”. He argued that the evidence for a miracle is rarely sufficient to suspend rational belief because a closer look has always revealed that what was reported as a miracle was more likely false, resulting from misperception, mistransmission, or deception….

A careful examination confirms that enthusiasm to emulate Mississippi should be tempered with scepticism….

In short, the authors followed a key point of logic: If something seems too good to be true, then it is likely not true.

In their analysis, On education miracles in general (and those in Mississippi in particular), they focused on two of the key problems with the story about Mississippi’s outlier grade 4 reading scores (in the top quartile of state scores) on NAEP: What is the cause of the score increases? And, why are Mississippi’s grade 8 reading scores remaining in the bottom quartile of state scores?

They found, notably, that Mississippi’s instructional reform, teacher retraining, additional funding, and reading program changes were not the cause of the score increases, concluding:

But it was the second component of the Mississippi Miracle, a new retention policy, perhaps inspired by New Orleans’ Katrina disaster a decade earlier, that is likely to be the key to their success….

Prior to 2013, a higher percentage of third-graders moved on to the fourth grade and took the NAEP fourth-grade reading test. After 2013, only those students who did well enough in reading moved on to the fourth grade and took the test.

It is a fact of arithmetic that the mean score of any data set always increases if you delete some of the lowest scores (what is technically called “left truncation of the score distribution”)….

In short, Mississippi has inflated grade 4 NAEP scores, but that is unlikely evidence that student reading proficiency has improved. This is not a story about reading reform, but about “gaming the system”:

It is disappointing, but not surprising, that the lion’s share of the effects of the “Mississippi miracle” are yet another case of gaming the system. There is no miracle to behold. There is nothing special in Mississippi’s literacy reform model that should be replicated globally. It just emphasises the obvious advice that, if you want your students to get high scores, don’t allow those students who are likely to get low scores to take the test. This message is not a secret….

Wainer, Grabovsky and Robinson’s analysis also needs to be put in context of two other studies.

First, their analysis puts a finer point on the findings by Westall and Cummings, whose comprehensive review of contemporary reading reform found the following: Third grade retention (required by 22 states) is the determining factor for increased test scores (states such as Florida and Mississippi, who both have scores plummet in grade 8), but those score increases are short-term.

Next is a recent study on grade retention. Jiee Zhong concluded:

[T]hird-grade retention significantly reduces annual earnings at age 26 by $3,477 (19%). While temporarily improving test scores, retention increases absenteeism, violent behavior, and juvenile crime, and reduces the likelihood of high school graduation. Moreover, retained students exhibit higher community college enrollment but lower public university attendance, though neither estimate is statistically significant.

Grade retention masquerading as reading reform, then, is fool’s gold for inflating test scores, but it is also harming the very students the reform purports to be helping.

The evidence now suggests that reading reform should not be guided by miracle claims; that no states should be looking to a miracle state for reading reform templates; that the so-called “science of reading” movement is mostly smoke and mirrors, and should be recognized as the “science of retention”; and that grade retention policies are distorting test scores at the expense of our most vulnerable students in life changing ways.

[1] Thomas, P.L. (2016). Miracle schools or political scam? In W.J. Mathis & T.M. Trujillo, Learning from the Federal Market-Based Reforms: Lessons for ESSA. Charlotte, NC: IAP.

If you want to help with the costs of keeping my public work open access and free, please DONATE.