Mathematica Policy Research has good news for KIPP. Their students make significant gains. The press release follows with a link to the report and summary of the findings.

A few questions occur to me about the replicability of the KIPP model.

First, KIPP has raised hundreds of millions of dollars from philanthropists and the U.S. Department of Education. Does that extra money translate into smaller classes and other perks? If not, what is it used for?

Second, to what extent do KIPP students benefit from peer effects, in that the comparison group is attending schools with kids with more problems and issues than those in KIPP?

Third, will KIPP ever take on the challenge of an entire small impoverished (I call it “the KIPP Challenge”)? So long as they take some but not all, the suspicion of selective attrition and exclusion will linger.

Here is the press release.

New Report Finds KIPP Middle Schools Produce Significant Achievement Gains

Contact: Jennifer de Vallance, (202) 484-4692

WASHINGTON, DC—February 27, 2013—A report released today by Mathematica Policy Research shows that Knowledge Is Power Program (KIPP) middle schools have significant and substantial positive impacts on student achievement in four core academic subjects: reading, math, science, and social studies. One of the report’s analyses confirms the positive impacts using a rigorous randomized experimental analysis that relies on the schools’ admissions lotteries to identify comparison students, thereby accounting for students’ prior achievement, as well as factors like student and parent motivation. Fact sheet.

Key findings on KIPP’s achievement gains include:

KIPP middle schools have positive and statistically significant impacts on student achievement across all years and all subject areas examined. In each of their four years of middle school, KIPP schools produced positive academic impacts on state standardized tests. Significant positive impacts are evident on average as well as for the majority of individual KIPP middle schools in the study.

The magnitude of KIPP’s achievement impacts is substantial. In each of the four subjects studied, KIPP schools produced achievement gains large enough to have a substantial impact on student outcomes:

Math: Three years after enrollment, the estimated impact of KIPP on math achievement is equivalent to moving a student from the 44th to the 58th percentile of the school district’s distribution. This represents 11 months of additional learning growth over and above what the student would have learned in three years without KIPP.

Reading: Three years after enrollment, the estimated impact in reading is equivalent to moving a student from the 46th to the 55th percentile, representing 8 months of additional learning growth over and above what the student would have learned in three years without KIPP.

Science: Three to four years after enrollment, the estimated impact in science is equivalent to moving a student from the 36th to the 49th percentile, representing 14 months of additional learning growth over and above what the student would have learned in that time without KIPP

Social Studies: Three to four years after enrollment, the estimated impact in social studies is equivalent to moving a student from the 39th to the 49th percentile, representing 11 months of additional learning growth over and above what the student would have learned in that time without KIPP.
The matched comparison design produces estimates of KIPP’s achievement impacts similar to estimates of the same impacts based on an experimental, lottery-based design. Researchers found that KIPP’s achievement gains are similar for the matched comparison design and the experimental lottery analysis.

KIPP’s gains are not the result of “teaching to the test.” For KIPP students in the lottery sample, researchers administered the TerraNova test—a nationally norm-referenced test—which students had not prepared for, and which carried no consequences for students or schools. The impacts shown in the TerraNova test were consistent with those shown in state tests.
Mathematica senior fellow and study director Philip Gleason said, “KIPP is making important strides to close achievement gaps for disadvantaged students. Findings from this large and comprehensive evaluation show that KIPP schools lead to educationally meaningful increases in student achievement, not just in basic reading and math, but in a broader set of subjects, including science and social studies.”

In addition to studying academic impacts, researchers also administered surveys to students and parents in the lottery group, to assess how KIPP affects behavior and attitudes toward school. The surveys showed that KIPP students complete up to 53 minutes more homework per night than they would have at non-KIPP schools, and that winning a KIPP lottery had a positive effect on both parents’ and students’ satisfaction with school. However, they also found that KIPP students reported no discernible increase in attitudes associated with success, and had an increased incidence of self-reported undesirable behaviors, including losing their temper, arguing with or lying to their parents, or giving their teachers a hard time.

The new report—the latest from Mathematica’s multi-year study of KIPP middle schools—is the most rigorous large-scale evaluation of KIPP charter schools to date. The report confirms and adds to the findings of the first Mathematica report on KIPP schools, released in 2010. The newly released 2013 report covers twice as many schools: 43 KIPP middle schools in 13 states and in the District of Columbia. In addition, the new report includes a broader range of student outcomes, examining not only state test results in reading and math, but also test scores in science and social studies; results on a nationally normed assessment that includes measures of higher-order thinking; and behaviors reported by students and parents.

The report also describes the population of students entering KIPP schools. Researchers found that students entering KIPP schools are similar to other students in their neighborhoods: overwhelmingly low achieving, low income, and nonwhite. Ninety-six percent are either black or Hispanic, and 83 percent are eligible for free or reduced-price school meals. Before enrolling in KIPP, typical students had lower achievement levels than both the average in the elementary school they attended and the average in the district as a whole. On the other hand, KIPP students are somewhat less likely than others in their elementary schools to have received special education services or to have limited English proficiency.

About Mathematica: Mathematica Policy Research, a nonpartisan research firm, provides a full range of research and data collection services, including program evaluation and policy research, survey design and data collection, research assessment and interpretation, and program performance/data management, to improve public well-being. Its clients include federal and state governments, foundations, and private-sector and international organizations. The employee-owned company, with offices in Princeton, N.J.; Ann Arbor, Mich.; Cambridge, Mass.; Chicago, Ill.; Oakland, Calif.; and Washington, D.C., has conducted some of the most important studies of education, disability, health care, family support, employment, nutrition, and early childhood policies and programs.