The National Education Policy Center urges caution when reading the CREDO study of charter schools in New Orleans. Governor Bobby Jindal has already taken CREDO as evidence for the success of privatization.
NEPC says not so fast. In addition to technical issues in the study, the critics make the following observations:
“Even setting aside these concerns, the effect sizes reported for New Orleans—let alone for the state as a whole—are not impressive in terms of absolute magnitude. Differences of 0.12 standard deviations in reading and 0.14 in mathematics indicate that less than one half of one percent of the variation in test scores is explainable by membership in a charter school.
“The study’s methods raise concerns that the findings could easily be misinterpreted to inflate pro-charter conclusions. In context, there’s little to crow about: the results from Louisiana and New Orleans are not much different from the uninspiring national results; the results for the state’s suburban charter schools showed negative gain scores (somewhat less growth in charters than in the comparison schools); and the small positive results reported for New Orleans are confounded by the devastating aftermath of a unique disaster.”
An even more serious challenge to the study was posed by New Orleans-based “Research on Reforms,” which complained that the Louisiana Department of Education will not release student data to independent research organizations.
It wrote: “As long as the Louisiana Department of Education can determine to whom to release student records for research purposes, the reports produced thereof, such as the CREDO report, are nothing more than biased evaluations.”
“The Department of Education (DOE) maintains that it has the discretion to release de-identified student-level records to selected researchers, and that it has the discretion to deny the same student records to other researchers. And, for the past few years, that is what the DOE has done. CREDO received the student records, and, Research on Reforms, Inc., who submitted a public records request for the same student records, was denied. As long as the DOE gets to select its evaluators, i.e., its researchers, the impact of the state-takeover and the charter school movement will never be objectively evaluated.
“Specifically, the Department of Education (DOE) released de-identified student-level records to CREDO for the school years 2008-09, 2009-10, and 2010-11 and denied the student level records for the same school years to Research on Reforms, Inc. (ROR). Thus, ROR sued the DOE in October 2012 for violation of Louisiana’s Public Records Act. The matter is now in Civil District Court.”
Reblogged this on Crazy Crawfish's Blog and commented:
This is said better than I could say so many thanks to Diane Ravitch, NEPC, and ROR and the researchers who critiqued this incredible CREDO, “study”.
Reading that the state is getting choose its own researchers reminds me of something I have encountered when exploring educational research on a couple of topics. Robert Slavin at Johns Hopkins produced the Success for All whole school program (and its parallel Roots and Wings). At one point when I checked for the research literature on the programs I noticed a disturbing trend – all of the studies had as lead authors Slavin or his wife Nancy Madden and the other authors listed had apparently been their students (I am doing this from memory of more than a decade past).
I also got interested around the same time in value added assessment. When I went to check the professional literature at that time on the Tennessee Value-Added Assessment System, with one set of exceptions all the literature had Sanders name on it. The exception were two studies commissioned by the Tennessee State Auditors Office:
Bock, R. Darrel, Wolfe, Richard, and Fisher, Thomas. “Review and Analysis of the Tennessee Value Added Assessment System.” (part one) Tennessee: Office of Education Accountability (Nashville, TN), 1996.
Fisher, Thomas H. “A Review and Analysis of the Tennessee Value-Added Assessment System.” (part two) Tallahassee, FL: Florida Department of Education: 1996.
In those days, TVAAS was a black box, the inner workings of which Sanders refused to disclose, although the mandate from the State Auditor gave the authors of the studies access not at that time available to others.
I remember that at one point Eric Hanushek had done a major presentation of why additional money did not make a difference in educational outcomes. His was a meta-study, with some real design issues. What was interesting is that when two economists at Princeton analyzed exactly the same date on which Hanushek had depended, they came up with very different results – sorry do not remember which of the Allens (Krueger or Sinai) that was.
Side note – worth mentioning that the original CREDO study of charters was run by Margaret Raymond, who is the wife of Erik Hanushek. Don’t know what relevance that may carry, but thought those who were unaware should at least be cognizant of the relationship.
There are many problems with research methods. They start with the fundamental assumption that it is appropriate to compare district & charter public schools. Both groups of schools vary widely.
Seems to some people, including me, that it is as meaningful to compare district & charter test scores as mileage of rented & leased cars. Not meaningful. Here’s a brief statement about that:
http://www.startribune.com/opinion/50494982.html
Okay I have some concerns. When the CREDO came out and said Florida Charter school students lagged behind public school students, I ran with it and told everybody I could that despite huge advantages charter schools weren’t performing better and that should be a huge red flag. But now we are questioning the CREDO? I want the right answer, not for my answer to be right.
Chris, CREDO research has many critics. That includes people who want to see charters go away and people who think the charter idea can be useful (thought it’s not the solution to the problem).
The NEPC reviews of CREDO studies, including the “original” 16-state study, have consistently raised these concerns. See http://nepc.colorado.edu/thinktank/review-multiple-choice
NEPC = NEA. There is no empirically based policy work coming out of this organization. I also question their journalistic standards when they simply republish blog posts from its members, regardless of the appropriateness of their tone or underlying content.
gee, you must not be paying attention to the credentials and expertise of the independent scholars/experts who perform the evaluation. You don’t like their conclusions so you try an ad hominem attack that fails miserably.
The NEPC “think tank review” panel of experts is available for review here: http://nepc.colorado.edu/think-tank/review-panel
These are indeed among the nation’s top researchers in these education policy areas.
The policy center publishes a great deal of empirically based policy work. Our publications are found here: http://nepc.colorado.edu/publications
As stated on our website, the NEA is among our supporters (see http://nepc.colorado.edu/support, where others are very much encouraged to provide support!). We’re very happy with the NEA’s strong understanding and agreement with our requirements for complete editorial freedom. (The same is true of our other funders, including our largest funder, the Ford Foundation.)
Even if we were so inclined — and we are not — we could never call up a professor at Stanford, Berkeley, UCLA, etc and say, “Hi. Our overlords at the NEA would like for you to write something that will support their positions.”
Regarding our “best of the ed blogs” feature (which is what has apparently cheesed you off), the goal is definitely not to endorse (or reject) the bloggers’ ideas. Rather, the goal is to bring to our readers’ attention provocative and informative new posts — which is the basic idea of blogging. That feature, btw, is found here: http://nepc.colorado.edu/blog.
There is one other major flaw in the CREDO study of Louisiana charter schools. Several of the charter schools in New Orleans used in the CREDO study are highly selective charter schools that function much the same way as academic magnet schools. Yet in the study these schools are also compared to regular public schools. This greatly distorts the results for the entire charter group. I believe that if the students from these schools had been compared to similar students in public magnet schools instead of all inclusive schools, all of the minor advantages of charters in Louisiana would have disappreared. But since the data used is kept secret by our DOE, there is no way for independent researchers to check the CREDO results.
I’m putting Pauline Lipman’s book on my reading list along with Diane’s new book.
The New Political Economy of Urban Education: Neoliberalism, Race, and the Right to the City
Pauline Lipman
May 13, 2013 – Education
Urban education and its contexts have changed in powerful ways. Old paradigms are being eclipsed by global forces of privatization and markets and new articulations of race, class, and urban space. These factors and more set the stage for Pauline Lipman’s insightful analysis of the relationship between education policy and the economic, political, and ideological processes that are reshaping cities in the United States and around the globe.
Using Chicago as a case study of the interconnectedness of neoliberal urban policies on housing, economic development, race, and education, Lipman explores larger implications for equity, justice, and “the right to the city”….Her synthesis of these lenses gives added weight to her critical appraisal and hope for the future, By examining the cultural politics of why and how these relationships resonate with people’s lived experience,
Mor