Two members of the honor roll–both thoughtful, dedicated educators–disagree about Néw York’s plan to evaluate educators, in this case principals.

Carol Burris, the principal of South Side High School in Rockville Center, was selected by her colleagues as principal of the year in Néw York. Mike McGill is superintendent of schools in Scarsdale, one of the state’s most affluent and excellent districts.

I honored Carol in the past for leading the fight against the state’s ill-considered test-based evaluation plan. I honored Mike for his stalwart opposition to the state’s demand to make testing the centerpiece of its vision and for his vision of what good education is.

Here, Mike takes issue with Carol’s critique of the state plan to evaluate principals. He thinks she didn’t go far enough in resisting a mindless technocratic bureaucracy bent in stamping out the last vestige of professionalism and independent thought.

Mike McGill writes:

Why the New York Value-Added Measure of Principals is Flawed (Part II)

New York Principal of the Year Carol Burris has been pushing back against the misuse of metrics in teacher evaluation. Now, in a letter to the Board of Regents, she’s taken on the Value-added Method (VAM) that’s being used to calculate 25 percent of principals’ performance rating.

I have concerns about the state’s approach as well, but I have to admit that I feel a bit ambivalent about her going public with hers. More on that in a minute.

Ms. Burris is concerned that Albany is going to measure principals on an uneven field. She says their scores will be calculated unfairly: Individuals’ ratings will reflect the performance of very different student populations that take different tests whose rigor differs.

She also worries about unintended consequences. Will schools advise students to avoid more challenging courses so their scores will be better?

Will they drop distinctive local programs so more students can take more state tests, so principals will have a better chance of getting better scores? Will principals in troubled schools leave and go where student populations are more stable, problems are fewer, and results better?

I’m not sure which unhappy outcomes are most likely, but I can’t imagine that the state’s plan will be especially productive in the end.

So why am I ambivalent about Ms. Burris’s message? It’s a matter of being careful about what you wish for.

Having observed the Albany mindset in action over the years, I find my own thoughts eliding quickly to another unintended consequence.

If, as Ms. Burris says, inconsistent measurement is the problem, there’s an easy solution. To be sure all principals are rated the same way, we could just make sure all schools in the state offer exactly the same program so that all kids take exactly the same tests.

Evaluation will drive instruction with even more of a vengeance.

The approach would be a little extreme, and to be fair, even our friends upstate might not want to go that far. Still, the technocratic impulse is to see complex difficulties as technical problems and then to solve them with mechanical fixes. And where schools are concerned, that impulse can lead to places nobody should want to venture, at least if he or she is interested in an innovative and distinctive education. More regimentation isn’t a prescription for excellence.

Okay. My comment about being ambivalent was a little tongue-in-cheek. But my experience here in the self-proclaimed “State of Learning” does give me pause. So just in case it might sound as if there’s a simple technical solution to the problems in Albany’s evaluation plan, let me offer four other reasons there isn’t.

One: VAM is supposed to compensate for the fact that different teachers or principals serve different populations.

So, for example, it compares those who work primarily with English Language Learners with others who do too. But VAM doesn’t distinguish among many other less obvious conditions that influence children’s learning. So in theory, it may level the playing field for people who work with different populations. In the real world, it doesn’t necessarily.

Two: Mathematical models can identify individuals whose students have progressed more or less on state tests. But that doesn’t mean that the student “output” can be attributed primarily to a particular person’s “input” in any particular case. The preponderance of research continues to indicate that statistical bias and random “noise” in the data skew VAM calculations and make them unreliable. We also know from experience that VAM results are unstable; for no evident reason, someone who’s a “high performer” this year may be a “low performer” next.

Three: Principals can’t control students’ or teachers’ actions tightly enough to be directly accountable for state test scores. For example, what if a new principal’s faculty is full of internal tensions, veterans are burned out or a significant number of students see school as irrelevant? She can’t unilaterally change work rules or conditions. She can’t fire tenured people for being apathetic. She has to work with the students she has. Realistically, how accountable can she be for achieving good VAM results, especially if she’s only been in the school for a short time?

Four: Value-added is only part of the state’s evaluation formula. A lot of the rest of a principal’s score depends on observations and other evidence. Supervisors are supposed to use objective criteria to score this evidence. (“The principal can express an educational vision. The principal holds meetings where he shares his vision,” for example.) Unhappily, however, this approach de-emphasizes capacities like the ability to use good judgment or to work well with people. Those qualities elude statistical measurement, must be judged subjectively, and don’t fit the evaluation model very well. Of course, they’re also among the most important things effective leaders do in the real world.

Those are four reasonable concerns about the premises underlying the state’s principal evaluation scheme.

But will anyone in Albany care?
In the world of education today, policy makers and practitioners stare at one another across a broad divide.

Basically, they’re working from different systems of belief. Many out here in the field say the theory that drives current policy is disconnected from reality. Our counterparts in state capitals and Washington tell us they know best and that we’ll just have to stay the course.

The way out of this unproductive tableau is through authentic dialogue. But that means those in the seats of power must want to listen.