The Washington Post has a story today reporting that the “median” charter school in D.C. outperforms the “median” public school.
This claim is based on a study that shows a difference in proficiency rates between the two sectors.
There are two ways, at least, to read this story:
One is that the leadership of the D.C. public school system is failing, that the “reforms” of the Rhee-Henderson era are a bust because the only way they can help students is to get rid of them, to push them off into the charter sector, because the leaders of the public school system have no clue about how to “reform” public schools.
The other is that changes in proficiency rates, as Matthew Di Carlo has demonstrated time and again, are meaningless (see here too). See also this post, where he showed how Mayor Bloomberg refers to proficiency rates as “meaningful,” but also “arbitrary.” They are indeed arbitrary. Read Andrew Ho’s important paper on how arbitrary proficiency rates are.
Di Carlo writes:
…… district officials and other national leaders use rate changes to “prove” that their preferred reforms are working (or are needed), while their critics argue the opposite. Similarly, entire charter school sectors are judged, up or down, by whether their raw, unadjusted rates increase or decrease.
So, what’s the problem? In short, it’s that year-to-year changes in proficiency rates are not valid evidence of school or policy effects. These measures cannot do the job we’re having them do, even on a limited basis. This really has to stop.
The literature is replete with warnings and detailed expositions of these measures’ limitations. Let’s just quickly recap the major points, with links to some relevant evidence and previous posts.
- Proficiency rates may be a useful way to present information accessibly to parents and the public, but they can be highly-misleading measures of student performance, as they only tell you how many test-takers are above a given (often somewhat arbitrary) cutpoint. The problems are especially salient when the rates are viewed over time – rates can increase while average scores decrease (and vice-versa), and rate changes are heavily dependent on the choice of cutpoint and distribution of cohorts’ scores around it. They are really not appropriate for evaluating schools or policies, even using the best analytical approaches (for just two among dozens of examples of additional research on this topic, see this published 2008 paper and this one from 2003);
- The data are (almost always) cross-sectional, and they mask changes in the sample of students taking the test, especially at the school- and district-level, where samples are smaller (note that this issue can apply to both rates and actual scores; for more, see this Mathematica report and this 2002 published article);
- Most of the change in raw proficiency rates between years is transitory – i.e., it is not due to the quality of a school or the efficacy of a policy, but rather to random error, sampling variation (see the second bullet) or factors, such as students’ circumstances and characteristics, that are outside of schools’ control (see this paper analyzing Colorado data, this one on North Carolina and our quick analysis of California data).
Diane Ravitch's blog 
Diane & Blog Readers: Get this! K12 is now invading Illinois suburbs–they are trying to open virtual charters in the Fox Valley area schools. There has been a charter applicant for the suburb of Naperville, a suburb which is highly regarded for its EXCELLENT schools! (Don’t know, as yet, if that charter applicant is K12–I have asked the person who sent out the alert to contact Diane as she, of course, rightly predicted suburban invasion.)
BEWARE: coming soon to a suburb near you…
I hate to sound callous, but a part of me says good. It’s about time for the affluent suburbanites to get off their behinds and join the action.
Yes, that is when resistance to the edu-predators will gain momentum; when it is not limited to “other schools.”
I actually sent Diane and email regarding this proposal. An umbrella organization has proposed a virtual charter to 18 suburban districts. Illinois has set up a new State Charter Commission that has the power to override a districts no vote on setting up the Charter school, in this case a virtual charter pulling from the 18 districts. Any fight against this will need to be done at the state level. Public meetings are scheduled for next week in most of the districts and then Virtual Learning Solutions can appeal any no votes by the Boards of Education. They must then hold public meetings with the State Commission. More info here http://beaconnews.suntimes.com/18695630-417/fox-valley-districts-set-hearings-on-proposed-virtual-education-effort.html
Whenever a state legislature creates a state commission appointed by the governor to override local school boards that veto charter school, you can be sure of one thing: The legislature is copying ALEC model legislation.
On the face of it, this report trashes the supposed success Michelle Rhee brought to DCPS.
Reblogged this on dayamonay.
I have learned that you know nothing about an individual school or school district without a complete 10 year spreadsheet. This revealed that LAUSD has lost over 10 years 156,000 enrollment and student who do not come to school everyday which is the difference between enrollment and ADA has gone from 2% or 14,500 to 17% or 112,000. If you do not do these spreadsheets how would you know these important facts of the districts total failure. You can talk all day long, You need the documented facts. LAUSD has over 10,000 fewer teachers. When you compare the teachers who no longer have a job at LAUSD and the lost students as a percentage of losses they are within .7% and that is fairly close. If you do not have students to teach you do not need teachers and staff for those who are not there. This is on top of the teachers they fire with false charges and illegal firings without the proper Skelly Hearings and Due Process.