Thomas Ultican investigated CREDO, the Stanford-Hoover organization that specializes in the study of charter school performance.
Ultican reviews the origin and history of CREDO and concludes that its long association with conservative and libertarian funders and groups influence its conclusions. He maintains that its methodology is flawed and biased to favor charter schools. He argues that its findings are meant to support privatization of public schools.
CREDO has long been an apologist for the New Orleans charter schools. The Louisiana state Department of Education has provided them exclusive access to select data so other research groups can not analyze CREDO findings.
This is a wonderful critique of CREDO’s research. Thank you Tom (and Diane).
The most baffling part of CREDO’s research is the use of standard deviations in test scores with fictional equivalents of “days of learning” lost (or gained). Anyone who has taught in public schools knows that states set required days of attendance, which has nothing to do with days of learning. Moreover, if learning occurs, it is not limited to tested subjects–usually two in the CREDO studies, math and reading.
The idea of fictional (virtual) twins serving as a control groups is another amazing case of conflating some elegant array of calculations with an authentic warts-and-all control group.
As an undergraduate,1954, I read a wonderful little book with great illustrations called How to Lie with Statistics. I recommend it for anyone who wants a one-hour refresher on statistics. I see that a pdf of the original, a bit fuzzy, is available at http://faculty.neu.edu.cn/cc/zhangyf/papers/How-to-Lie-with-Statistics.pdf
The most widely read peer-reviewed article by Eric Hanushek has the title “ The Economics of Schooling: Production and Efficiency in Public Schools (1986). It was written while he worked at Ronald Reagan’s Congressional Budget Office. Schooling is treated as if not radically different from a factory–inputs, outputs and so on. A pdf of this article is online.
Thank you for the wonderful remarks. I will definitely look at your recommended book. You have added some real value to my article here.
In the article, Tom: quotes ““Advanced statistical techniques are employed to disentangle the influences on achievement of schools and teachers from those of other factors such as family backgrounds and student abilities.”
Such techniques didn’t exist then and don’t exist now. You cannot use statistics to analyze nonquantifiable subjects, such as “student achievement”, “teacher influence”, “student abilities”. Just because those people who claim that these things are quantifiable are very loud, powerful and repeat their propaganda ad nauseum, doesn’t make these things describable by statistics.
Exact sciences endlessly debate what such basic concepts such as time and distance are, yet we allow exceedingly complicated terms such as “knowledge”, “good teacher”, “bad student” go undefined in “scientific” papers.
Many people don’t want to look into such papers because they are full of complicated formulas, and they don’t feel competent to judge the merits of the paper. But teachers do know that complicated and how unquantifiable education is, hence looking into such paper what they should search for is not the meaning of terrifying formulas but what the formulas claim to describe and analyze. Ask questions like “what do the authors mean by student achievement?” or “what do they mean by teacher effectiveness?” or “what do they mean by knowledge?”, and if they don’t find answers to these questions, there is no reason to assume, the paper has anything to say about education.
I passed almost all of this article along in current affairs at the Post Dispatch. The title: “Credo has St. Louis on its list..Blythe Bernard:don’t fight them–the pd will fire you”…..I start by explaining…”Blythe Bernard is the education writer for the Post Dispatch. I have watched education writers come and go….one thing has remained constant over the last 15 years….those who refuse to be cheerleaders for the charters do not last very long.”