The National Education Policy Center reviewed CREDO’s latest report on ranking charter organizations and found it wanting.
CREDO Report Fails to Build Upon Prior Research in Creating Charter School Classification System
Key Review Takeaway: Report overstates its findings, ignores relevant literature, and fails to address known methodological issues, suggesting an agenda other than sound policymaking.
NEPC Review: http://nepc.colorado.edu/thinktank/review-CMOs
Report Reviewed: https://credo.stanford.edu/pdfs/CMO FINAL.pdf
Contact:
William J. Mathis: (802) 383-0058, wmathis@sover.net
Gary Miron: (269) 599-7965, gary.miron@wmich.edu
Learn More:
NEPC Resources on Charter Management Organizations
BOULDER, CO (September 7, 2017) – Charter Management Organizations 2017, written by James Woodworth, Margaret Raymond, Chunping Han, Yohannes Negassi, W. Payton Richardson, and Will Snow, and released by Center for Research on Education Outcomes (CREDO), assessed the impact of different types of charter school-operating organizations on student outcomes in 24 states, plus New York City and Washington, D.C. The study finds that students in charter schools display slightly greater gains in performance than their peers in traditional public schools, especially students in charter schools operated by certain types of organizations.
Gary Miron and Christopher Shank of Western Michigan University reviewed the report and found CREDO’s distinctions between organization types to be arbitrary and unsupported by other research in the field. This raises concerns about the practical utility of the CREDO findings.
In addition, Miron and Shank contend that CREDO researchers made several dubious methodological decisions that threaten the validity of the study. A number of these problems have been raised in reviews of prior CREDO studies. Specifically, CREDO studies have been criticized for:
Over-interpreting small effect sizes;
Failing to justify the statistical assumptions underlying the group comparisons made;
Not taking into account or acknowledging the large body of charter school research beyond CREDO’s own work;
Ignoring the limitations inherent in the research approach they have taken, or at least failing to clearly communicate limitations to readers.
These problems have not only gone unaddressed in Charter Management Organizations 2017, but have been compounded by the CREDO researchers’ confusing and illogical charter organization classification system. As a result, the reviewers conclude that the report is of limited value. Policymakers should interpret the report’s general findings about charter school effectiveness with extreme caution, but might find CREDO’s work useful as a tool to understand how specific charter school management organizations perform relative to their peers.
Find the review, by Gary Miron and Christopher Shank, at:
http://nepc.colorado.edu/thinktank/review-CMOs
Find Charter Management Organizations 2017, by James Woodworth, Margaret Raymond, Chunping Han, Yohannes Negassi, W. Payton Richardson, and Will Snow, published by CREDO, at:
https://credo.stanford.edu/pdfs/CMO FINAL.pdf
The National Education Policy Center (NEPC) Think Twice Think Tank Review Project (http://thinktankreview.org) provides the public, policymakers, and the press with timely, academically sound reviews of selected publications. The project is made possible in part by support provided by the Great Lakes Center for Education Research and Practice:
The National Education Policy Center (NEPC), housed at the University of Colorado Boulder School of Education, produces and disseminates high-quality, peer-reviewed research to inform education policy discussions. Visit us at: http://nepc.colorado.edu
Diane Ravitch's blog 
I am so glad this is being addressed by researchers. These studies have constantly tried to obscure the data rather than to make it transparent.
And despite claims that CREDO takes attrition into account, there has never been a good explanation as to how this is done. We are just supposed to take their word for it.
One charter: 100 students start. The lowest performing are weeded out quickly with a few getting through the cracks who are held back so that they aren’t in the cohort being compared anymore.
One public school: 100 students start. They leave when their parents move away or find a better school for their child.
If CREDO wants to be seen as an honest operator, they will spend a week to put together all this research they supposedly have and give us a ranking of charter schools from the best to the worst. Every single one. They can do it by state. Or city. And then wait for the outcry from the education reformers who suddenly realize how dishonest and misleading CREDO’s claims to be able to make these comparisons really are.
LikeLike
“They leave when their parents move away or find a DIFFERENT school for their child.”
There corrected!
LikeLike
Thank you! Correction was needed.
LikeLiked by 1 person
CREDO has always had a pretty obvious pro-charter bias, but it doesn’t matter because it’s always had the veneer of pure objective academic research. Truthiness started long before the Trump administration.
LikeLike
Am looking for another cell phone provider. Guess CREDO is a NIX. CREDO comes across as being liberal. Guess CREDO is more on the fascist side.
LikeLike
The National Education Policy Center states “The findings in CREDO’s report do not offer sufficient education-relevant benchmarks by which to evaluate the magnitude of their observed results.” They found at least six different flaws in their research methodology. This is to say, if this study were submitted as a legitimate scholarly study, it would be filed in the round file. The Waltons specialize in selling junk, and this time they paid for some junk science.
LikeLike
“Over-interpreting small effect sizes”. Decades of over-interpreting small effect sizes! And all based on standardized test junk science, for that matter. Junk in, junk out.
LikeLike
Is this the study that was commissioned by the Walton family?
LikeLike
Walton is the major funder of CREDO at Hoover
LikeLike