Carol Burris, executive director of the Network for Public Education, wrote in The Progressive about the role of the conservative Hoover Institution as a reliable advocate for charters, vouchers, and all kinds of school choice. (When I was a conservative, I was a Senior Fellow at Hoover.)
Burris writes:
One of the original intentions of creating charter schools was to improve student learning—which is why it’s telling that proponents of “school choice” now justify charter-school and school-voucher expansion by saying they are necessary to provide parents with options other than traditional public schools.
Choice for choice’s sake—originally a secondary rationale for charters—has become the go-to line of charter school proponents. Meanwhile, measures of academic performance have faded into the backgroundas a justification for school options. Nevertheless, for years, the question of whether or not charter schools academically out-perform traditional public schools has gnawed at the industry like an annoying uncle who insists on having the last word in every family debate.
The latest attempt to prove the supposed superiority of the charter industry comes from the Center for Research for Education Outcome, or CREDO, which has taken prior stabs at the question with results that were far from convincing.
“Remarkable” was how Margaret “Macke” Raymond, CREDO’s director and author described the results of CREDO’s latest national charter school study. Her enthusiasm was infectious. The Wall Street Journal’s editorial board proclaimed that CREDO’s new evidence showed charter schools are now “blowing away their traditional school competition.”
But despite the headlines that popped up in pro-charter media, the only thing “blown away” was the truth. Like prior national studies, CREDO’s latest report, “As a Matter of Fact: The National Charter School Study III,” shows tiny average differences between charter and public school students—0.011 standard deviations in math and 0.028 standard deviations in reading. These are differences so small that the Brookings Institution’s Tom Loveless once likened them to standing on a few sheets of paper to increase one’s height.
And CREDO knows it. The organization characterized nearly identical differences in their 2009 national study as inconsequential—referring to them as “meaningless,” “small,” and possibly derived from “measurement error.”
How could “meaningless” suddenly become “remarkable” once a tiny statistical tilt in outcomes favors charter schools? The answer lies in who runs CREDO, who funds it, and the methodological problems inherent in its reports.
We tackle these points in our new Network for Public Education report, “In Fact or Fallacy? An In-depth Critique of the CREDO 2023 National Report.” Here’s a brief summary.
Who runs CREDO?
Although reporters refer to CREDO at Stanford University or Stanford’s CREDO, the relationship between CREDO and the prestigious university is complicated.
CREDO is based in the conservative, pro-charter Hoover Institution, a private think tank on the Stanford University campus. The Hoover Institution governs and finances itself without oversight or control by the university. In fact, Hoover has a “long and fraught relationship” with Stanford’s faculty and students who have objected to its lack of diversity, controversial scholarship, and conservative ideology.
What all of these funders have in common is a vested interest in charter schools and—at least in Pearson’s case—profit.
Expanding school choice is a focus of the Hoover Institution. For example, in 2021, Hoover hosted Betsy DeVos in a stop on her book tour. Secretary DeVos was introduced and praised by Raymond, who, along with her role at CREDO, refers to herself as the education program director at Hoover in the video.
Please open the link and read the rest of the article.

“The Hoover Institution governs and finances itself without oversight or control by the university….
Which just happens to host the Hoover institute on its campus.
But other than that, no relationship whatsoever.
It appears that there was some “demand (threat?) for clarification” by Stanford after Diane’s last post on CREDO
Ha ha ha ha
LikeLike
Or maybe their objection was to Diane’s post on the forced resignation of their president?
LikeLike
What an embarrassment it must be to work for Stanford these days.
LikeLike
When institutions take the money, they must accept the criticism from hosting such a right wing group as the Hoover Institution. By associating with Stanford University, Hoover gains a connection to legitimacy and gravitas without actually earning either.
LikeLike
And the Stanford board of trustees get to air their right wing views with plausible denial.
So everybody wins
Hooverford”
Hoover is Stanford
And Stanford is Hoover
Despite a demand for
Epoxy remover
LikeLike
CREDO’s web address is Stanford.edu.
Richard Hanania is scheduled to teach a seminar at Stanford. Hanania’s attitudes about women and Jews were exposed by Huffpo’s investigative reporting last week.
Then, there’s the issue of the untimely resignation of Stanford’s president. I presume Stanford, a think tank with students, views integrity in a way that few others do.
Serving manna for themselves and the wealthy.
In 2021, Raymond’s husband, the Hoover Institute’s Hanushek, got the “world’s largest prize in education” from a Chinese industrialist.
If there’s a hell, may all involved with ed reform and all right wing tech kings and discount retailer heirs rot in it.
LikeLike
“’Remarkable’” was how Margaret ‘Macke’ Raymond, CREDO’s director and author described the results of CREDO’s latest national charter school study.
[…]
“And CREDO knows it. The organization characterized nearly identical differences in their 2009 national study as inconsequential—referring to them as ‘meaningless,’ ‘small,’ and possibly derived from ‘measurement error.'”
I think that’s a little unfair.
In 2009, CREDO actually stated both this:
“In reading, charter students on average realize a growth in learning that is .01 standard deviations less than their TPS counterparts. This small difference — less than 1 percent of a standard deviation — is significant statistically, but is meaningless from a practical standpoint.”
but also:
“In math, the analysis shows that students in charter schools gain significantly less than their virtual twin. Charter students on average have learning gains that are .03 standard deviations smaller than their TPS peers. Unlike reading, the observed difference in average math gains is both significant and large enough to be meaningful. In both cases, however, the absolute size of the effect is small”
And it is plausible that what Raymond deemed “remarkable” was the charter school improvement reflected between the 2009 and 2023 studies which at 23 to 24 days difference exceeded what in 2009 CREDO considered both significant and meaningful.
Raymond also, or alternatively, may have deemed particularly “remarkable” the gains of certain subgroups:
“In the current study, we find that Black students in poverty had 37 days stronger growth in reading and 36 days stronger growth in math when compared to their TPS peers. The results were similar for Hispanic
students in poverty: they grew 36 more days in reading and 30 more in math than their TPS peers.”
While one doesn’t readily discern it from NPE’s critique these are not gains over the course of the schooling, but purportedly annual, one year’s gains on top of another, year after year, is that correct?
Multiple choice:
CREDO’s conflict of interest is to NPE’s conflict of interest as:
a) a mountain is to a molehill
b) plant-based butter is to sour cream
c) none of the above
d) sheesh, is this test almost over yet?
LikeLike
Stephen,
The “days of learning” measure is not considered legitimate by other scholars. It magnifies small differences.
LikeLike
Diane: “The “days of learning” measure is not considered legitimate by other scholars.”
Ah, so I went to bard.google.com and inquired whether the “days of learning” measure used by CREDO at Stanford is considered legitimate by other scholars
And it informed me:
“According to Tom Ultican, ‘days of learning’ is a method unique to CREDO and generally not accepted by scholars.”
While Bing’s AI stated: “The “Days of Learning” measure used by CREDO at Stanford is considered legitimate by other scholars.”
So I went to my current fave AI, Anthropic’s Claude, which safely responded:
I do not have enough context to definitively evaluate the legitimacy of the “days of learning” measure used by CREDO at Stanford. However, here is some relevant information:
CREDO (The Center for Research on Education Outcomes) is a research center at Stanford University that studies the effects of education reforms and policies.
One of the metrics CREDO uses in some of its research is “days of learning”, which aims to quantify the learning gains or losses for students under different conditions. For example, comparing learning gains for students in charter schools versus traditional public schools.
The “days of learning” approach converts assessment score gains into the equivalent number of days of learning required to achieve that gain. This provides an intuitive way to compare impacts across different contexts.
While some researchers consider this a useful metric, others have critiqued it as oversimplifying the complexities of learning and exaggerating small test score differences. There are also questions around the validity of the specific conversion formulas used.
Overall there is no consensus among academics on whether this is a fully legitimate and appropriate measure. As with any metric, its strengths and limitations need to be considered. The broader education research field uses a diverse range of methods and measures.
So in summary, while CREDO stands by its use of “days of learning”, there are differing perspectives on its merits and validity as a measure. There does not appear to be unanimous agreement or a definitive conclusion on its legitimacy from the broader research community. As with any research, the methods should be scrutinized and limitations considered.
LikeLike
STEPHEN.
The fact that CREDO is funded by pro-charter Waltons, John Arnold, and Reed Hastings should give you pause. The three have poured hundreds of millions into charter schools. Kind of like the tobacco industry publishing research proving that cigarettes are good for you.
LikeLike
But isn’t losing the daze of learning a good thing?
LikeLike
“There does not appear to be unanimous agreement or a definitive conclusion on [days of learning] legitimacy from the broader
researchAI community.”Fixed
What a tangled maze we wend when first we quote the LLM
AI yie yie
LikeLike
Stephen. You appear not to have read our report. Whether “days of learning” is legitimate or not is inconsequential to the argument I make in the report. What is consequential is what small effect sizes they represent. The largest you cite, 37 days, only represents 0.065 standard deviations– but that is not even large enough to be considered a small effect size.
Jacob Cohen, a psychologist and statistician, is best known for his work on effect sizes. Cohen categorized effect sizes as small if they meet the thresholds of 0.2, medium at 0.5, and large if it reaches 0.8 standard deviations.
When CREDO was critiqued on this point in 2017, this was their response: “…the IES [Institute of Educational Sciences] report points to an example effect size of 0.15 on reading achievement scores as “an effect quite likely to be judged to have practical significance in an elementary school context.”
What you cite is less than half of what the IES says has “practical significance”
In this new report, CREDO states, “Across the three CREDO national charter school studies, annual charter student learning in reading has risen by 22 days; math learning has increased by 23 days.” These report-over-report increases (where the baseline results showed charters at a deficit) are also small, representing 0.038 and 0.04 of a standard deviation, again well below the 0.15 effect size CREDO cites as having “practical significance.”
And by the way, CREDO changed its methodology. Read the report, which cites what serious scholars have said time and again about CREDO reports.
Macke and her husband are school choice advocates. They are not impartial researchers. Her doctorate is in political science, yet she was listed as the first author of the report.
Here is where NPE and CREDO differ. We are honest. We clearly state we are a public school advocacy organization and are critical of what charter schools have become. CREDO hides its advocacy. It should not have access to confidential student data given to it by states, with no ability for scholars to check their work.
I suggest you add option e. “I need to actually read the reports before having an opinion.”
LikeLike
Stephen Ronan– “The ‘days of learning’ approach converts assessment score gains into the equivalent number of days of learning required to achieve that gain.”
Anyone with common sense and/ or classroom experience knows that there is no norm for how many days of “learning” [classroom instruction] it takes one individual student vs another to learn a concept [= “achieve that gain”]. The construct is completely undermined when the supposed “gain” [learning a concept] is measured by an occasional fill-in-bubble standardized test.
LikeLike
Carol-
Have you posted your concerns at PubPeer?
LikeLike
“The “days of learning” measure is not considered legitimate by other scholars. It magnifies small differences.”
Some (not I , of course) would say that is the whole purpose.
But in my opinion, they are just cynics.
What could one possibly hope to gain by magnifying an ant by a power of 180 ? (number of school days)
So, the ant would go from 1 cm to 1.8 meters. No biggy. Everyone knows there are actually 1.8 meter ants in Africa (called Giant African ants, appropriately). So it’s not like it’s anything unusual.
LikeLike
“unfair”
We’re talking Stanford! The school’s President just resigned after years of scrutiny of his research. Critics call SIEPR, the Stanford Institute for the Evisceration of People’s Retirement. Stanford hired Joshua Rauh after the scathing criticism his public pension papers received while he was at Northwestern.
Speaking of Rauh, his is the seminar following Richard Hanania’s at Stanford’s Classical Liberalism Initiative in October. Huffpo reported about Hanania in an investigative post last week. Who funds the Initiative? Why do the 3 men listed at CLI’s site, I presume they’re in charge (they’re from Hoover), have bio’s without grants or outside income sources listed?
On the spectrum of university and think tank with students, where would you place the Stanford/Hoover outfit?
Stephen Ronan, my multiple choice question
You want to get credible information, do you choose
(A) an institution with a reputation for integrity
(B) a right wing-affiliated, mutated, universityish campus with ties to anti-democracy tech kings
(C) never Stanford
LikeLike
We’ve been told to answer “C” when we don’t know a multiple guess question. . .
So, I answer E–None of the above.
LikeLike
Wait, we’re not allowed to answer a question with our own logical answer. How about if I answer with a question: Who gives a damn?
LikeLike
“The ‘days of learning’ approach converts assessment score gains into the equivalent number of days of learning required to achieve that gain.”
Anyone with common sense and/ or classroom experience knows that there is no norm for how many days of “learning” [classroom instruction] it takes one individual student vs another to learn a concept [= “achieve that gain”]. The construct is completely undermined when the supposed “gain” [learning a concept] is measured by an occasional fill-in-bubble standardized test.
LikeLike
Ginny, thank you for that spot-on takedown of the totally bogus “days of learning” metric based on actual teacher experience, which none of these think tank guys seem to have. Think tank: a place where thinking tanks. The standardized tests given in ELA are completely invalid, and the ones given in math are imbalanced. So, that’s another reason why the “days of learning” metric is bullshit.
LikeLike
So, Pearson, which takes its pound of flesh these days by selling testing to public schools, funds CREDO, which works to undermine public schools.
Hmmmmm.
LikeLike
Follow the money!
LikeLike
“Remarkable” was how Margaret “Macke” Raymond, CREDO’s director and author described the results of CREDO’s study
I agree.
Simply reMackeble.
LikeLike
CREDO is reMackeble
CREDO is reMackeble
Utterly in CREDObull
CREDO is unpackable
Simply inconceivable
LikeLike
All of the people listed at CREDO’s site, including Margaret Macke Raymond have Stanford.edu e-mail addresses. None have Hoover email addresses.
LikeLike
Linda
Doncha know?
Lots of people with White House emails don’t work at the White House, at least not now.
Vladimir Putin, for example.
LikeLike
Thanks for the laugh
LikeLike
The long history of charter school advocacy suggests that improved student learning was never the goal. That was just the talking point to justify the real goals, privatization of public schools and in many cases segregation. The obfuscation was needed because most Americans want public education and reject public funds for private and religious schools. Of course, not true about every charter school or every parent who sends their child. However, it characterizes current advocacy. It’s overwhelmingly bad policy for equality and democracy.
LikeLike
Thanks, Arthur. It’s important to remember that the charter school idea was born in the 1950s as a ploy to send public funding to white segregation academies.
LikeLike
We Americans keep falling for the same old bait and switch routine.
LikeLike
yup
LikeLike
“When I was a conservative, I was a Senior Fellow at Hoover.
Now there is an acceptable, proper usage of the term conservative!
LikeLike
And come to find out, you weren’t regressive enough, hence your being let go.
LikeLike
She obviously wasn’t fellow enough.
Not one of the good ole fellows.
LikeLiked by 1 person
Fellow Fell Low
Diane was fellow
But not enough
Fellow who fell low
Below the snuff
LikeLiked by 1 person
Diane was fellow
But not enough
Fellow fell fallow
Below the snuff
LikeLike
Hoover: where men are men and women are fellows.
LikeLike
Does Hoover allow marriage between fellows?
LikeLike
Fellowships, as it were.
LikeLike
Herbert Hoover, one of out worst presidents pretty much used his alma mater, Stanford, as a bullhorn to criticize Franklin D Roosevelt, one of our greatest presidents. Hoover changed the Hoover Institute in the 1950s from a war library to a conservative thinking tanked and an attack on the New Deal and the working class, and that it remains to this day. It is a legacy of a failed president who sat on his hands to lead his country —and his alma mater into the Great Depression. CREDO comes from a long line of Hoover clunkers. The Hoover Institute is like having a James Buchanan Institute at Dickinson College that still criticizes Lincoln and the 13th Amendment. Stanford would have been more fortunate today had it rejected Herbert Hoover’s application in 1890.
LikeLike
It’s kind of fitting that Stanford still hosts Hoover institution because the Stanford officials have the same elitist attitude toward “ordinary” Americans that Herbert Hoover had.
LikeLike
One of Hoover’s visiting “research fellows” is Michael Petrilli of Fordham-a political science major with absolutely no background or qualifications to engage in research. That speaks volumes about the Hoover Institution.
LikeLike
You tell ’em, Carol. And thank you for the outstanding work that you do!
LikeLike
PubPeer for analysis of CREDO and Fordham research?
LikeLike
Replying to you here, Carol, to escape the narrowing column….
And thanks, while I had quickly perused the entirety of the NPE report shortly after it appeared, I’d appreciate anything you can do to help me understand that material better.
When you alluded there and here to Clark’s purported representation of what constitutes a small, medium or large effect size, my immediate, inexpert response was puzzlement over how one could possibly assert a uniform system of such categorization regardless of what’s being measured. And, indeed when I go now to the CREDO rebuttal that you alluded to, it appears you’re missing important context when you write, “What you cite is less than half of what the IES says has ‘practical significance.'”
To quote CREDO more completely:
“Miron and Shank criticize that most of the effect sizes found in the CMO study do not merit attention. They then cite Cohen’s example benchmarks of 0.2, 0.5, and 0.8 for small, medium, and large effect sizes respectively. However, right after introducing these thresholds, Miron and Shank then point out that Cohen himself states those thresholds were only examples and are not appropriate for all areas of study. In fact, the Institute of Education Sciences (IES) report (Lipsey 2012) cited by Miron and Shank states, ‘Cohen’s broad categories of small, medium, and large are clearly not tailored to the effects of intervention studies in education.’ Rather, the IES report points to an example effect size of 0.15 on reading achievement scores as ‘an effect quite likely to be judged to have practical significance in an elementary school context.’ Further, the report goes on to point out that a similar effect at the high school level would be huge. Since it is reasonable to presume Miron and Shank read the entirety of the report they cited, it is left to conclude they are fully aware of the misleading nature of their claim that the effect sizes found by CREDO are trivial simply because those sizes fall below Cohen’s suggested ‘small’ effect size.”
Click to access response-to-nepc-critiques-of-credos-cmo-study.pdf
If, Carol, you imagine CREDO’s use of “huge” may exaggerate, you’d find this in Lipsey:
“An effect size of, say, 0.15 on reading achievement scores for that intervention will, therefore, represent about a 25 percent improvement over the annual gain otherwise expected for second-graders—an effect quite likely to be judged to have practical significance in an elementary school context. That same effect size for tenth graders, on the other hand, would nearly double their annual gain and, by that benchmark, would likely be viewed as a rather stupendous effect.”
When your report alludes to a chap standing on sheets of paper do you think that most of your report’s readers would intuit that for that particular arithmetic exercise just 3 inches was being considered a full standard deviation?
In respect to the “37 days stronger growth in reading and 36 days stronger growth in math when compared to their TPS peers,” that’s the average for each year of attendance, right?
So if we imagine a student attending a charter school for grades 6 to 12 one might anticipate that their test results were consistent with having benefited from about year or more of additional schooling compared to TPS counterparts? Is that right?
You would regard such gains as unremarkable?
How about this from an earlier CREDO study:
“Charter students in Boston gain an additional 12 months in reading and 13 months in math per school year compared to their TPS counterparts.”
Click to access mareportfinal_000.pdf
Note: “per school year”. I confess when I first encountered that I thought it must be in error and remain open to such possibility.
LikeLike
Researchers have used Cohen’s effect sizes as a guide for years. Vouchers decrease learning by -.40 SD. Effect sizes exceeding .20 are not uncommon. You are drawing all kinds of conclusions that cannot be drawn. CREDO has never claimed these are year-after-year cumulative gains for students. The Lipsey quote (comparing elementary student gains to high school gains) furthers the argument that many have made against Days of Learning–learning gains are not standard across school years. (Read the critiques).
Regarding Boston–2013 study. 2023 Massachusetts in the new report 41 DOL So, assuming Boston is similar to the rest of the state, that is a drop from 365 to 41. (41 by the way is an effect size of only 0.07.) Boston charter schools are notorious for over-the-top discipline that pushes kids out.
The bottom line, if you look at the preponderance of studies, including the 2019 study using NAEP scores by the National Center for Education Statistics, you find that charter school students’ achievement is similar to their peers in public schools. Scholars with no advocacy positions when it comes to charter schools have said the same thing over and over about CREDO for years–bad methodology and over-exaggeration of results.
There is a reason that the only national newspaper that covered their report was Murdoch’s Wall St. Journal.
Look at the individuals praising CREDO on their website. Every one of them a proponent of school choice.
Robert Enslow? He runs the organization Milton Friedman founded.
LikeLike
Stephen, Carol doesn’t have time to continue replying to you, so I’m ending your challenges to her.
LikeLike
The only thing I have good to say about Hoover Institute is the longitudinal meta-study of millions of test scores 1971-2017 reported in summer 2022 by their publishing arm, Education Next, entitled “A Half Century of Student Progress Nationwide.” The report shows that hischool grads over those yrs have 4 more yrs of math content under their belt, & 1 more yr of reading content. They even go into the fine details of why reading ability is not likely to increase at the same rate as math ability. Worth a read. https://www.educationnext.org/half-century-of-student-progress-nationwide-first-comprehensive-analysis-finds-gains-test-scores/
LikeLike
The National Charter School Study III,” shows tiny average differences between charter and public school students—0.011 standard deviations in math and 0.028 standard deviations in reading“
About some things we can be certain:
Time will pass
Entropy will increase
And mathturbators will mathturbate
LikeLike
Stanford’s Classical Liberalism Initiative is hosting a seminar by Richard Hanania in October. Hanania’s Center for the Study of Partisanship and Ideology is located in the Salem Center at the University of Texas.
Media report that the Salem Center was able to expand because of Mr. Salem’s efforts at funding. Mr. Salem’s Linked In profile tells us he is involved with ACE scholarships (connected to the Waltons) and he is “an active member of a national network funded by Charles Koch…connecting, providing resources and educational opportunities to social entrepreneurs.”
Hanania posted an interview with Amy Wax, “The feminization of the Academy has been a total disaster.”
LikeLike
Some women researchers will betray the national interest to serve wealthy Republicans. Some Black people will do the same. They get a small piece of the pie for their betrayal. But, the White, rich, GOP guys always, hold, overwhelmingly the highest positions.
Review the demographics of the boards and top management of the right wing- funded centers listed in the above comment at 9:03.
LikeLike