New York is very proud of its new Educator Effectiveness Evaluation model, which claims to measure which teachers and principals are effective, relying in part on the increase (or not) of test scores of students.

Bruce Baker of Rutgers demonstrates that the model is biased and inaccurate. It favors classes and schools that start off with higher-performing students.

He concludes with a brief sermon about the importance of ethics:

“I have pointed out that the originators of the SGP approach have stated in numerous technical documents and academic papers that SGPs are intended to be a descriptive tool and are not for making causal assertions (they are not for “attribution of responsibility”) regarding teacher effects on student outcomes. Yet, the authors persist in encouraging states and local districts to do just that. I certainly expect to see them called to the witness stand the first time SGP information is misused to attribute student failure to a teacher.”

“But the case of the NY-AIR technical report is somewhat more disconcerting. Here, we have a technically proficient author working for a highly respected organization – American Institutes for Research – ignoring all of the statistical red flags (after waiving them), and seemingly oblivious to gaping conceptual holes (commonly understood limitations) between the actual statistical analyses presented and the concluding statements made (and language used throughout).”

“The conclusions are WRONG – statistically and conceptually. And the author needs to recognize that being so damn bluntly wrong may be consequential for the livelihoods of thousands of individual teachers and principals! Yes, it is indeed another leap for a local school administrator to use their state approved evaluation framework, coupled with these measures, to actually decide to adversely affect the livelihood and potential career of some wrongly classified teacher or principal – but the author of this report has given them the tool and provided his blessing. And that’s inexcusable.”

Bruce Baker is one of my favorite bloggers. He is smart and irreverent. He is not awed by big names. He actually was a teacher before becoming a researcher. He has the technical skill to crack the statistical analyses that others generate to make spurious claims. Unlike many with the same skill set, he is willing to call a phony a phony. You might say he is our Premier C.D. (Since I have taken a personal pledge not to use vulgarisms on this blog, I will give you a clue to help you figure out what a C.D. is; it is a Cr-p Detector).

In this post, Bruce shows what nonsense The New Teacher Project report “The Irreplaceables” is.

TNTP, you may recall, was founded by either TFA or Michelle Rhee, depending on whom you heard from last. Its purpose was to stock urban districts with shiny new teachers (like TFA) to replace those burned-out veterans with low expectations. Although TNTP is an advocacy group that seeks and wins contracts from urban districts, and although it has a self-interest in certain policies, it nonetheless turns out studies and reports to support its self-interest (fire bad old teachers, hire and retain new teachers). It is not surprising that advocacy groups crank out self-serving “studies,” but it is surprising that the media so often takes them seriously, sort of like taking advice from tobacco companies about the wisdom of smoking.

In his analysis of the latest bit of TNTP puffery, Baker demonstrates how unstable value-added measures are. He reviews the ratings for thousands of NYC teachers and finds that there are very few who remain in the top 20% every single year. In fact, he discovers only 14 in math and 5 in ELA out of thousands of teachers! He writes, “Sure hope they don’t leave.”