Gary Rubinstein learned that KIPP plans to add more schools in Philadelphia and nearly triple its enrollment.
So Gary did what he always did: he checked the public data for KIPP in Philadelphia.
He found that KIPP has one high school in the city.
“That school got the lowest possible rating, essentially an F. Not only were their test scores low, but they also got the lowest possible rating in ‘growth’ in math and reading, in other words the value-added for the school which reformers claim to take very seriously.”
But, says KIPP spokesmen, an amazing proportion of our students graduate from college.
Gary checked the data. KIPP was resorting to its usual legerdemain and ignoring the high attrition from fifth grade to high school graduation. They compare all low-income kids only to their high school graduates, which makes KIPP look far better than the reality.
Given KIPP’s unimpressive academic record–one might say “failing” record–in Philadelphia, why would the School Reform Commission allow them to expand?
Diane Ravitch's blog 
“They compare all low-income kids only to their high school graduates, which makes KIPP look far better than the reality”
Quibble: I think instead of “their high school graduates” you’d need to state “their 8th-grade graduates” to properly reflect Gary’s assertion.
Fifteen years ago, Caroline Grannan exposed this, and it’s been repeatedly exposed ever since. Here’s Norm’s notes and then Richard Kallenberg from 2011
http://normsnotes2.blogspot.com/2011/01/myths-and-realities-about-kipp.html
x x x x x x x x x x x x x x x x
Myths and realities about KIPP
Kahlenberg looks at attrition rates in Bay Area – which Caroline Grannan first exposed. Though KIPP studies like the one by Mathematica shows that there may be high attrition rates at the public schools w/ which KIPP was compared, these mostly low-performing kids go back into the public school system elsewhere, whereas KIPP does not replace them w/ other low-performing students.
The school filters them out the most problematic students over time, so only the best students remain in smaller classes over time. The peer effects are likely substantial… something not explored in any of the charter school studies.
Also, as in most charter school comparisons, there are many demographic differences between public and charter schools that these studies do not control for: homelessness, severe disabilities, and the % of students that are free lunch (usually they only compare % students that are free and reduced price lunch, lumped together, with the latter group having an entirely different level of expected achievement.)
http://voices.washingtonpost.com/answer-sheet/charter-schools/myths-and-realities-about-kipp.html?wprss=answer-sheet
x x x x x x x x x x x x x x x x x x x x x x x x x x
Myths and realities about KIPP
By Valerie Strauss
This was written by Richard D. Kahlenberg, a senior fellow at The Century Foundation. He is the author of “All Together Now: Creating Middle-Class Schools through Public School Choice.”
In this post, he refers to a debate that I had with my inimitable colleague Jay Mathews about school reform that discussed Teach for America and the Knowledge Is Power Program, or KIPP. You can find that debate here.
By Richard D. Kahlenberg
In the recent education debate between Valerie Strauss and Jay Mathews, a question arose about the attrition rates at the highly regarded Knowledge Is Power Program (KIPP) schools. The issue is important because if large numbers of weaker students drop out of KIPP’s rigorous program, it would be highly unfair to compare the test score gains won by the top KIPP students against the scores of all regular public school students – who include KIPP dropouts.
In the debate, Strauss mentioned some studies finding that KIPP schools “have had a very high attrition rate.” Mathews responded by saying it is a “myth that KIPP schools have poor retention rates” and cited a 2010 study that found that KIPP school “are doing about as well as regular schools in their neighborhoods” in terms of attrition.
Who’s right? While I respect Jay Mathews’s grasp of educational issues, on this question, the data overwhelmingly support Valerie Strauss’s skepticism.
In a rigorous 2008 study of five KIPP schools in the San Francisco Bay Area, researchers at SRI International found that an astounding 60% of KIPP students left over the course of middle-school. Moreover, the researchers found evidence that the 60% of students who did not persist through the tough KIPP regimen (a longer school day and week, and heavy doses of homework), tended to be the weaker students.
KIPP supporters, like Mathews, respond that a 2010 study of 22 KIPP schools by Mathematica found that the attrition rates were comparable to nearby high poverty public schools that also have lots of kids leave. Poor people tend to move frequently, so high attrition rates are to be expected at KIPP schools, it is argued.
The big difference between KIPP and regular public schools, however, is that whereas struggling students come and go at regular schools, at KIPP, student leave but very few new children enter. Having few new entering students is an enormous advantage not only because low-scoring transfer students are kept out but also because in the later grades, KIPP students are surrounded only by successful peers who are the most committed to the program.
Below is a figure that shows the attendance at KIPP Bay-area schools. (The figure is part of a Century Foundation document entitled “Charter Schools that Work: Economically Integrated Schools with Teacher Voice.”)
Bay Area KIPP Net Student Enrollment by Grade from 2003-04 to 2006-07
studentenrollment.jpg
In the comments section of the Answer Sheet blog, when readers pointed out that KIPP schools don’t generally fill students back in, Mathews responded “KIPP schools DO take in new students beyond the 5th grade.”
This is technically accurate, but as the figure above suggests, the vast majority of students enter during the 6th grade (a natural time to enter middle school) and then the total number of KIPP students in 7th and 8th grade falls precipitously.
The KIPP Bay-area schools cannot be dismissed as an outlier on the KIPP attrition question. Columbia University researcher Jeffrey Henig’s 2008 review of several studies found high attrition rates at a number of other KIPP schools.
It may well be, in fact, that high attrition rates are a key explanation for KIPP’s success in raising test scores. When KIPP tried to take over a regular public school – where the students are not self-selected, but are assigned to the school; and where students not only leave, but large number of students enter — KIPP abandoned the field after just two years. KIPP long ago realized that what we charge regular public schools with doing is far more difficult than what KIPP seeks to do.
I’m not clear on it, but I read a lot of ed reformers and they say these “100%!” numbers they use are valid because of “repeaters”
I’m not sure what that means but it sounds like students who repeat a grade? So the reason they lose half the class between 6th grade and 12th grade is because the students are repeating grades, and will eventually graduate?
Maybe Gary Rubenstein could explain what it is they’re talking about with this. Has he heard this “repeater” thing?
I’m not certain what exactly you’re alluding to in respect to “100%!” numbers but I think the info you’ll find via the link below is at least tangentially relevant to your queries here, Chiara:
https://dianeravitch.net/2017/07/08/u-s-news-kicks-kipp-nyc-school-off-its-list-of-best-high-schools-due-to-gary-rubinsteins-analysis/#comment-2702780
The other justification ed reformers use for cherry picking and attrition is “magnet schools”
They say there are public magnet schools so it shouldn’t matter if charters cherry-pick.
But ed reform never portrayed charter schools as magnet schools. In fact, they insist on a direct comparison to ordinary public schools- that was the point- they could do what public schools do BETTER.
If they’re just a variety of magnet then that’s a completely different argument.
We don’t have a magnet public school here but if we did no serious person would compare the ordinary public high school to a selective public high school because that’s a silly comparison. Of course the selective does better. That’s the point of the magnet school- it’s selective.
If ed reformers argued that there should be charter MAGNET schools we could debate that, but they never have- instead they insist they are “just like” public schools.
Excellent point, Chiara.
Has anyone bothered to look at how PUBLIC schools have done under ed reform?
Why is this such a low priority? It’s incredible to me that there’s NO INTEREST in the public schools in these “choice” cities.
President Obama promoted a single Cleveland charter chain over and over and over. Never a mention of the public schools that SURROUND that charter chain. That’s more than an accidental omission- that’s a preference.
How can they say they value these schools equally if they don’t care at all what happens to the existing public schools?
DeVos offers these ridiculous talking points- parents will have unlimited choices in a place like Van Wert Ohio and NO school will get hurt?
That’s a fantasy. If she fragments a system with 30,000 students in it ALL the schools will be weaker. Public schools already ARE weaker under ed reform in Ohio. They have harmed existing public schools.
Chiara: “Has anyone bothered to look at how PUBLIC schools have done under ed reform?
“Why is this such a low priority? It’s incredible to me that there’s NO INTEREST in the public schools in these ‘choice’ cities.”
On the contrary it is an exceedingly important issue that has been examined closely.
Take a careful look at this to see how Boston schools have done alongside a growing charter sector: https://www.bostonpublicschools.org/domain/238
It starts with this “Over the past twenty years, the Boston Public Schools (BPS) has been transformed from a failing school district to one of the most renowned urban public school systems in the country.” And then provides a whole lot of supporting detail.
And for a broader examination of the question, perhaps this would be helpful:
“The Effect of Charter Schools on Students in Traditional Public Schools: A Review of the Evidence” By Brian P. Gill 11/02/2016
“…As the table indicates, the literature provides some support for the ‘healthy competition’ hypothesis and almost none for the hypothesis that students in district schools are harmed by the growth of charters. Six studies found some evidence of positive effects, four found no effects, and one found negative effects. Breaking the results out by locations, in six cases that encompass five cities and states, there is evidence that charter schools produce (small) positive effects on the achievement of students in nearby public schools. In nine other cases, encompassing eight cities and states and one nationwide sample, charter schools have been found to have no effect on students in nearby district schools, positive or negative. The literature has only a single case—involving a single school district—in which charter schools have been found to have negative effects on the achievement of students in nearby district schools.”
http://educationnext.org/the-effect-of-charter-schools-on-students-in-traditional-public-schools-a-review-of-the-evidence/
Stephen,
You are aware, I assume, that EdNext is not a neutral source.
The editorial board (where I was once a member) is pro-charter, pro-voucher, funded by rightwing sources, affiliated with Hoover Institution.
Gill describes the research results in that EdNext article, but there’s no indication that that publication had any role in sponsoring the research or determining its outcomes. Gill and his colleague in the analysis, Kevin Booker, are at Mathematica Policy Research in Cambridge, Massachusetts. The research was published in the “Handbook of Research in Education Finance and Policy.” Gill states that “Dennis Epple, Richard Romano, and Ron Zimmer did a similar review in a recent paper, reaching conclusions similar to ours.”
I presume he’s referring there to “Charter Schools: A Survey of Research on Their Characteristics and Effectiveness” by Dennis Epple, Richard Romano, Ron Zimmer
NBER Working Paper No. 21256, June 2015. http://www.nber.org/papers/w21256
Edward Cremata and (your former colleague if I recall correctly) Margaret Raymond concluded as follows in “The Competitive Effects of Charter Schools: Evidence from the District of Columbia”:
“It is also clear that the competition provided by charter schools is multi-faceted and acts across various channels that this paper has just begun to explore. Our results also suggest that the interests of the charter and TPS sector may in fact overlap; a focus on quality in the charter sector may lead to positive competitive impacts in the TPS sector as well, a result less likely if the local charter sector is of middling quality. Given the occasionally contentious relationship between TPS and charter schools, and the significance this issue will gain as the charter market continues to grow, identification of mutual interests between the two sectors should be treated as welcome news, particularly to advocates of quality in the charter sector.”
Click to access
That sounds reasonable to me… While improvements in Boston TPS are largely attributable to ed reforms by Ed Reformers, they’re certainly not all attributable to charter schools. Those improvements are in large part parallel to, and compatible with, rather than deriving from, the charter school growth. But it seems likely that the high quality of the charter schools here helps to some degree.
As I concluded my college application essay long ago on Mary Jane: More research is needed.
Thanks for raising the important question, Chiara.
Stephen,
I was a member of the Koret Task Force at Hoover for 10 years. It is an elite group of conservative academics, including Paul Peterson, Eric Hanushek, Checker Finn, John Chubb, Terry Moe, and others. It was also the editorial board of EdNext, which Peterson led. The task force is very supportive of choice, charters, vouchers, accountability, testing, etc. I left it in 2009. Macke Raymond is at Stanford but not part of the task force. She is married to Rick Hanushek. I like them both very much, but not their politics.
Thanks. Interesting.
We can be confident he’s well aware of it, Diane; he’s just hoping the rest of us aren’t.
Love it! You’re defending EdNext by citing Mathmatica. Thanks, I needed a laugh!
“Love it! You’re defending EdNext by citing Mathmatica. Thanks, I needed a laugh!”
Enjoy! And then, if you have a moment, re-read what I wrote where there’s no particular defense or critique of EdNext. And, if you have an additional moment, perhaps offer a critique of the cited research. Or offer any studies that you consider more reliable. Thanks.
Incidentally, in case you were thinking otherwise, the Handbook of Research in Education Finance and Policy is not a peer-reviewed journal, but rather an ed reform advocacy book sponsored by the Association for Education Finance & Policy.
Rubinstein’s post is a crystal-clear illustration of attrition, & an object lesson in how to lie with statistics.
THE CLAIM: “the first 8th-grade class from KIPP’s original middle school, which graduated in 2007, boasted 35 percent of students obtaining four-year college degrees 10 years later, compared to a 9 percent rate for low-income tudents nationally.”
“…[That KIPP school had 33 8th graders in 2007 of which 12 graduated from college.] And the year before they had 55 7th graders. And the year before that they had 77 6th graders. And the year before that, that cohort had 86 5th graders. So 86 5th graders became 33 8th graders for an attrition rate of 62%. And those 12 students who graduated college, well they were not 35% of 35 but just 13% of the actual 86 students who entered that school.]
Charters crow about their wonderful “success.” The problem is they cherry pick data the way they cherry pick students. Instead of calling their 12th graders seniors, they should call them the survivors.
How amazing it is that this one teacher, in his spare time, is the only person who even QUESTIONED KIPP”s claims.
Hundreds of millions devoted to ed reform orgs, thousands of researchers, a whole segment of university professors who promote charter schools and not one person asked why KIPP is expanding?
Come on. That’s not a debate. It’s an echo chamber.
Proof that KIPP hasn’t found a secret sauce for helping the average inner city kid:
“When KIPP tried to take over a regular public school – where the students are not self-selected, but are assigned to the school; and where students not only leave, but large number of students enter — KIPP abandoned the field after just two years.”
It confirms what public school teachers have known for years. Poverty DOES matter!
The saddest part to me of watching 20 years of “free market” ed reform in Ohio has been watching my local public school try to comply with every demand, every fad, every talking point – I wish they would stand up for themselves and tell Fordham and the rest to go jump in a lake.
Ohio has changed the high school graduation scheme so many times to comply with the ever-shifting demands of the national ed reform “movement” I would be surprised if 1 in 100 public school parents understand it.
If they won’t get your input then stop dancing as fast as you can to gain their approval. There will never BE approval. This was never ABOUT “better” public schools. It was about replacing public schools. You can’t win this game.
We have had an over-subscribed vocational high school here for THIRTY YEARS.
I just listened to John Kasich insisting that we “stopped” offering vocational ed and he wants schools to immediately “start” offering it because vocational ed is currently fashionable in ed reform.
He completely ignores this giant 4 county public high school and insists it, I don’t know, doesn’t exist.
It’s just clueless. They often demand public schools should do things public schools are already doing, and have ALWAYS done.
Moratorium on charter chains now!
Amen, LeftCoast! Here in ILL-Annoy (where publics–especially Chicago–are being starved by taxpayer-funded charters) the new ed. budget bill has been stalled. Called “Fix the Formula,” it’s been pushed for at least 3 years by Advance ILL-Annoy* (& also, on the sly–& I say that because their name mysteriously comes & goes–ILL-Annoy Stand for Children) & some other of our favorite reform groups–Teach Plus, & Educators for Excellence. (Although, people keep telling me, “But there are a lot of other groups for it!” & “It will make IL children college & career ready!”**–now, where have we heard THAT mantra before?!). The threats are that, if this bill doesn’t pass (& it did pass both chambers, awaiting the governor’s signature), IL schools won’t open on time (this is coming from adminimals). Now, this “fix” might have been well & good–if only another bill–one to put a moratorium (& I believe–correct me if I’m wrong–for 10 years–?) on new charter school approvals/openings
–had been passed. But, nooo, THAT bill died in committee.
*Look up Advance in Laura Chapman’s excellent post (May 27th, I think), exposing 50CAN. (Yeah…50 “CAN”… close more public schools!)
**Shockingly enough (or, maybe not so much), the mantra came from a person on the financial end (& a good one), so these privatizers pull wool over the eyes of all, far & wide.
Well, except for us.
To answer the question of the post:
HELL NO!