A few days ago, I published a post about a paper by Kirabo Jackson, explaining that the non-cognitive effects of teachers are often more important than the test scores of their students.


As it happened, mathematician Robert Berkman read the paper and explains here why it is another nail in the coffin of value-added measures, which judge teacher quality by the rise or fall of student test scores.


Berkman writes:


In this post, I’m going to examine one of the studies that no doubt had a profound impact on the members of AMSTAT that led them to this radical (but self-evident) conclusion. In 2012, the researcher C. Kirabo Jackson at Northwestern University published a “working paper” for the National Bureau of Economic Research, a private, nonprofit, nonpartisan research organization dedicated to promoting a greater understanding of how the economy works (I’m quoting here from their website.) The paper, entitled “Non-Cognitive Ability, Test Scores, and Teacher Quality: Evidence from 9th Grade Teachers in North Carolina” questions the legitimacy of evaluating a teacher based on his/her students’ test scores. Actually, it is less about “questioning” and more about “decimating” and “annihilating” the practice of VAM.


He adds:


What should be noted is that Jackson is not an educational researcher, per se. Jackson was trained in economics at Harvard and Yale and is an Associate Professor of Human Development and Social Policy. His interest is in optimizing measurement systems, not taking positions on either side of the standardized testing debate. Although this paper should reek with indignation and anger, it makes it’s case using almost understated tone and is filled with careful phrasing like “more than half of teachers who would improve long run outcomes may not be identified using test scores alone,” and “one might worry that test-based accountability may induce teachers to divert effort away from improving students’ non-cognitive skills in order to improve test scores.”

But lets get to the meat of the matter, because this paper is 42 pages long and incorporates mind-boggling statistical techniques that account for every variable one might want to filter out to answer the question: are test scores enough to judge the effectiveness of a teacher? Jackson’s unequivocal conclusion: no, not even remotely.


The only puzzle is why Arne Duncan keeps shoving VAM down the throats of states and school districts.



PS: Berkman added his credentials in a comment:

“I’m a math teacher who has worked with pre-K through college aged students for 30 years. My degrees are in Urban Studies, and Elementary Math Education. I have also done extensive work in neuroscience and numeracy, as well as technology and education, not to mention cognitive science.”