Julian Vasquez Heilig of the University of Texas has the most brilliantly illustrated blog of any that I read. He creatively weaves in photographs, graphs, and other eye-catching stuff to make his text vivid.

And vivid it is.

In this post, he analyzes with his typical humor and dry wit the latest Mathematica study of Teach for America. The study made headlines across the nation.

It said that the students TFA’s young recruits got higher math scores than did the students of other novice teachers or of experienced teachers.

But Heilig demonstrates that the sample of TFA teachers was not typical of TFA, and that the differences between the TFA teachers and the other teachers were very small, almost to the point of being trivial.

He refers to the study as “irrational exuberance,” slyly referring to a remark made by Alan Greenspan when the stock market reached a feverish high in 1996. Greenspan implied that the market was a bubble, about to burst, and Heilig implies the same.

First, he points out that there are not many TFA secondary math teachers, and Mathematica had to “scour the country” to get an adequate sample size. Next, he notes that the sample was 80% white, which does not reflect the reality of urban districts or of TFA.

Most important, he shows how Mathematica chose to represent the differences; it used a scale showing the differences as “tenths of a standard deviation.” When the same difference is represented as 1, 2, or 3 standard deviations, it is very hard to see any difference between the novice TFA teachers and the experienced teachers. As he writes, “you need binoculars, maybe a telescope when the effects of secondary TFA math teachers are placed on a scale that is not in tenths of a standard deviation.” In other words, Mathematica presented the results in a scale that exaggerated what they found.

But his most remarkable observation is that the effect size of a TFA recruit is .07, while the effect size of class size reduction is .20. Thus, if you are a policymaker and you want to get the biggest improvement, you would reduce class size instead of hiring TFA, and you would get triple the effect!

Not to end with that huge finding, Heilig goes on to observe that the Mathematica study has findings that are “contrary to what we know from decades of research about teacher quality.”

According to Mathematica, nothing matters but “the magic of TFA.”

Prior ability in math doesn’t matter.

Taking math courses or have a math major in college doesn’t matter.

Working on your masters degree or certification has a negative effect.

Or, in the inimitable words of the irrepressible and brilliant Julian Vasquez Heilig:

In sum, you will be a better airline pilot (teacher) if:

  • You do not have ongoing pilot training, it will hurt your flying skills.
  • You do not study to become a pilot before piloting a plane. Just rev the engines. Wohoooooooo.
  • Using a flight simulator to test your ability to fly a plane before hand will have no relationship to your ability to fly a plane.

Please read the post and enjoy JVH’s irreverent and often hilarious graphics. Spot on.