This question comes up again and again, and different studies reach different conclusions. Typically, TFA teachers get better than usual results in math, but not in reading, which is less susceptible to test prep and more influenced by home life.

Mathematica Policy Research released a new study today, saying that TFA and TNTP teachers get better results in math than traditionally prepared teachers. But Dana Goldstein analyzes the findings and learns that the headline oversimplifies.

For one thing, the gains were modest: “For the average child in this study, who scored in just the 27th percentile in math compared to her peers across the country, having a TFA teacher will help her move up to the 30th percentile–still a long way off from grade-level math proficiency.”

For another, the study shows that experience matters: “The bias against first-year teachers is borne out in the data. The students of second-year teachers outperformed the students of first-year teachers by .08 standard deviations–a larger gap than the one between the students of TFA and non-TFA teachers. And even though TFA recruits did well in this study, that doesn’t mean teachers reach their pinnacle after two years on the job. To the contrary, the researchers found that for teachers with at least five years of experience, each additional year of work was associated with an increase of .005 standard deviations in student achievement. ”

And Goldstein notes that 89% of the TFA teachers in the study were white, which causes concern because there are many reports of urban districts losing teachers of color, especially African Americans. That may be as big or bigger a problem in the long run that a few percentage points up or down.