Governor Andrew Cuomo has come up with a compromise on the issue of releasing teacher data rankings. He wants only parents to see the rankings and data reports for their children’s teachers, but to make public the data for individual classes and schools. This is a step in the right direction, but it doesn’t go far enough.
It doesn’t satisfy the tabloids, who want every teacher’s name and ranking to be published online and in print. Like Mayor Bloomberg, they believe that the rankings are an accurate reflection of teacher quality and should be freely displayed, perhaps on “wanted” posters in the post-office.
It doesn’t satisfy me either because I know based on research and experience that the rankings are inaccurate, unstable and will sully the reputation of good teachers. How are parents helped by seeing inaccurate ratings? How are teachers helped to improve, as Bill Gates pointed out in an article in the New York Times earlier this year, if their job evaluations are showed to anyone other than their supervisors?
The rankings, derived from the rise or fall of student test scores, are demonstrably inaccurate. When New York City released its teacher data reports in January and they were published in the media with the names and rankings of teachers, it warned the public to take them with more than a grain of salt because the margin of errors in both reading and math were so large–35 points in reading and 53 points in math. That means that a teacher of math who was labeled a 50 (on a 100 point scale) might actually be at the 15th percentile or the 85th percentile. In reading, the margin of error was so large as to make the numbers utterly meaningless. Statistical analyses showed that there was no correlation between the scores that a teacher “produced” from year to year, and that a teacher who taught both subjects got different grades. All that data, all those rankings were so flawed as to be pointless other than to provide fodder for the tabloids to attack teachers.
Why aren’t the tabloids howling for the release of the evaluations of police officers and firefighters? Why are their evaluations shielded (by law) from public view? Shouldn’t the public have a right to know about their performance?
What about the job evaluations of the top officials at the New York City Department of Education? When will their job evaluations be released? They are public employees and they are paid six figures. What value do they add? How many schools have they improved? What are they doing to strengthen public education? How can the public hold them accountable? Here’s one suggestion: Every time a public school closes, the top officials should lose points on their evaluation because a school closing represents a failure of leadership.
Diane
Diane Ravitch's blog 
RE: private sector applies accountability to itself
Diane,
Earlier this Spring, I attending a ‘question and answer luncheon” with Gov. Bobby Jindal hosted by the Lafayette Chamber of Commerce. I was likely the only educator in the room and had plenty of questions to ask Gov Jindal but the Gov was a little late, spoke quickly and left the event before a single question was answered—-remember, this was as close to a tailored audience as the Gov could have hoped for and he still didn’t take a single question. During the Gov’s speech, I leaned over to a gentlemen sitting to my left who happened to be CEO of a hospital in the Lafayette area and asked him if he ever pondered posting his hospital’s mortality rate outside its door. Though I knew his answer would be ‘no’ I was a little surprised at how firmly his ‘no’ response was—-it was as if I asked him to jump off of a bridge. I was merely trying to make a comparison to cohort grad rates and letter grading systems in our state to the business community.
My point: accountability as educators know it will never be applied to any other type of profession much less within the business community despite their unwavering support of accountability for public schools. That CEO’s firm ‘no’ response was all the proof I needed that accountability the way we know it will not make anything better….and the business world knows this.
LikeLike
Your recollection and account of the Lafayette Q&A is absolutely priceless! Isn’t Gov. Jindal always in a hurry? Look how fast he moved the state’s ed reform through the legislature. And, if anyone has followed any of the events in recent months knows, he is never available for an eye-to-eye immediate response, unless he can say, “It’s unacceptable!” You know what, Bryon, maybe he had a plane to catch to go out of state to raise funds for another Republican or to promote his own personal agenda. Yes, Bryon, your comment is priceless and addresses the blog topic perfectly.
LikeLike
Bryan, had the CEO offered more than his terse response, I suspect his explanation would include that although doctors play a role in a patient’s health, there are a number of other factors that doctors have no control over–patient’s genetics, prior medical history, willingness to follow the doctor’s prescriptions, environment, how far an illness has progressed before the doctor sees the patient, etc.
And, of course, his explanation is perfectly valid. For some reason, though, when teachers make the same point regarding students’ test scores, corporate ed reformers are quick to accuse them of making excuses.
LikeLike
The last sentence in your comment about Cuomo’s compromise nails it:” Every time a public school closes, the top officials should lose points on their evaluation because a school closing represents a failure of leadership.” In NYC shouldn’t that be Mayor Bloomberg? And, with the endless round of school closings over the years shouldn’t he be fired at this point. It seems obvious to me that the only point of closing schools is to humiliate teachers in order to try and force veterans out to be replaced by new teachers at lower cost. I remember when I began teaching I asked veterans for help. When I became a veteran I was sometimes asked for help by new teachers. About seven or eight years ago veterans became “toxic” and new teachers were literally told to avoid veterans with their old “inside the box” methods by the Principal in our school! What about the damage done to the local educational communities when their schools are closed? How do students feel about attending and graduating from a “failing” school that is being phased out over a three year period? I see nothing good coming out of all these so called reforms. I cannot understand how some of our leaders can be so heartless and cruel in their efforts to humiliate teachers, destroy whatever pride students and parents may have in their “failed” schools and thoroughly destabilize the educational communities. I guess the profit potential is just too great in privatization to resist.
LikeLike
Bryan Alleman, you hit the nail on the head!!
LikeLike
It’s time to get rid of high-stakes testing. If we want to attract teachers to the profession then we should focus less on publishing student test scores and more on giving teachers the resources and ability to make their own assessments that will work for the students in their classrooms.
LikeLike
remember, a 95% CI means that 1 in 20 teachers fall outside the stated confidence interval (i.e. margin of error). And it’s even more if the assumptions of the model are violated (e.g. non-random assignment of students to teachers.
LikeLike
When Americans invest for retirement through a managed fund, 401K or pension, they pool their money over to portfolio manager who is supposed to follow the goals of the fund and invest wisely. Sometimes, the bank or investment firm will disclose the name of the portfolio manager, sometimes not.
In late 2008 and early 2009, the market collapse caused millions of Americans to lose a significant portion of their wealth. When these investors (people like you and me) logged into their retirements accounts and realized just how much their wealth had decreased, they didn’t immediately blame the portfolio manager. The portfolio manager simply invested in relevant securities as per the fund’s goals. American investors were smart: rather than blame the person simply doing his/her job, they looked beyond the visible symptom to the actual disease: large banks buying/selling toxic assets and large banks trading securities with risk they didn’t understand. Who allowed this to happen? Most of these banks fired their CEOs and Risk Management depts, or they “resigned” voluntarily.
The guys managing portfolios are still managing portfolios, doing their jobs by using the knowledge they acquired in college, work experience, etc. to help investors meet their goals. Education should be no different. We should be evaluating Superintendents, NYC’s “Education” Mayor, and all Secretaries of Education. Strategy, or policy wins a chess game, not individual pieces.
LikeLike
Schools are not stocks in a portfolio. We don’t keep the winners and sell the losers. Children are not investments that can be discarded if they don’t make the grade. We have to change our vision or lose our future.
Diane
LikeLike
Agreed. Unfortunately (and as you know) this is how “reformers” think, which explains why schools are being “discarded” as you so aptly put.
This is a tragedy. I can’t imagine what it must feel like in ten years for recent public high school graduates: to not be able to visit his/her old school because it was phased out, to not be able to meet up with his/her former teachers. No 10-year, 15-year, 20-year reunions. All in the name of test scores. That’s just sad.
LikeLike
Yo mista!,
“Most of these banks fired their CEOs and Risk Management depts, or they “resigned” voluntarily. ” Please cite the sources for this statement. Now, it’s my understanding that most of the CEOs are still there and have been still making a fortune with no repercussions whatsoever. Source please!
Thanks,
Duane
P.S. And I agree with Diane’s response-two completely different sectors of the human experience.
LikeLike
I must not have my tongue in cheek detector on!
LikeLike
Duane,
Very few are still there, many are not.
I used to work in one of these places between 2007-2009, but here are some examples:
– Dick Fuld, Lehman Brothers
– John Thain, Merrill Lynch
– Jimmy Cayne, Bear Stearns
– Ken Lewis, Bank of America
CEOs of Goldman and JPM still there as these guys didn’t harm their shareholders as significantly. Many top dogs involved in the terrible mergers during the crisis (B of A buying Merrill Lynch) were also asked to leave or severed.
Granted, these guys made a lot of money while working there and when they left/were fired. Forgetting the $ issue, people still blamed them (the CEOs) for the mistakes of the companies, and not individual traders, bankers, etc. Which is my point: you can’t blame individual teachers for bad policy, you have to blame the guys who make policy.
LikeLike
Thanks for your comments, interesting discussion. Has our society then, held those at the top (of the banking world) who make the policies which the underlings follow liable/responsible? Isn’t that what the ol Show Me Stater implied when he said “The buck stops here!” It doesn’t appear so to me.
And yes money does matter, as I’m still digging out of the housing/credit bubble, and I didn’t even come close to borrowing the supposed value of my house, didn’t even borrow the county appraised value and I still had to take tens of thousands of dollars in losses when I had to sell my place. So no, as far as I’m concerned heads should roll.
I have a tough time with not holding teachers responsible, not in the sense of test scores etc. . . but for the harm they know they are doing to the kids. I have yet to meet a teacher or administrator who thinks this whole testing regime is good but not many of them fight it because they are too afraid of losing their jobs-which actually is not an unfounded fear-what a way to run an organization, through fear. And that’s a tough position to take as so many teachers are only looking to make a simple living helping children learn, nothing extravagant and to do so requires following the insane mandates.
LikeLike
Duane,
You said, “I have a tough time with not holding teachers responsible, not in the sense of test scores etc…but for the harm they know they are doing to the kids. I have yet to meet a teacher or administrator who thinks this whole testing regime is good but not many of them fight it because they are too afraid of losing their jobs…”
I don’t agree with the testing regime either, and I am fighting it by writing about it. Unfortunately, teachers can’t fight the testing regime in their classrooms – it’s hard not to spend time on test-prep when in NY State every HS student must pass five high-stakes exams called “Regents.” Most teachers (that I know, including myself) would argue many of the topics on these exams are not that relevant to our students’ lives. Yet, these tests are the reason why some students don’t graduate on time, and principals can use Regents scores to have some pretty uncomfortable conversations with teachers.
The Regents themselves are actually very low in standard – one only needs to answer 15 multiple choice questions correctly (out of 39 total questions) to earn a scaled score of “65” (passing). If you do the math, what we’re really saying is a raw score of ~40% = content mastery.
So, my argument comes down to policy again. Tying the results of exams with anything important (money, graduation, etc.) almost always lead to cheating and/or lower standards. Teachers can (and should) fight the testing regime outside of the classroom, but not inside where they have to teach to what the national and state gov’t want them to. Teachers are good people, so they will do their jobs even if they don’t agree with the policy in place.
LikeLike
I blame our unions for negotiating us into this mess. We had a chance when the courts threw out the first APPR nonsense. We’ve been negotiated right up to the edge of the abyss. We’re not getting thrown into it, however we are about to be slowly lowered into it because of our unions concerns.
LikeLike
Cuomo had bullied NYSUT into a corner and they knew it. If they didn’t kunckle under and NYS lost the RttT “beauty contest” then his backers would have plastered the airwaves with anti-teacher commercials. Everything time he wants something (to burnish his 2016 Presidential “cred”) his moneyed minions do a slick ad buy across the state until he gets what he wants. Then Cuomo reneged on the agreement. What he didn’t get from NYSUT, he managed to go back and get the following year from the legislature after the RttT award was already approved What has not been reported is that so precious little of those millions of dollars is going to local districts to improve instruction. It’s all going to bureaucracy and more and more private testing operations. Cummo is despicable. A true anti-union DINO!
LikeLike
Your relentless exposure of the hypocrisy of making teachers accountable while giving a pass to all other professions is effective reasoning.
LikeLike
NYSED states the following on page 39 of the June 2011 New York State: Race to the Top Scope of Work – http://usny.nysed.gov/rttt/docs/statesow.pdf
“Performance bands will be set and the evaluation system will be designed in such a way that the majority of educators will be rated either “Developing” or “Effective.” The expectations for “Highly Effective” educators will be set appropriately high, so that the percentage of educators in this category will be relatively small.
For principals and superintendents, the new evaluation systems will represent a major change, and differentiating performance as New York State intends will require a cultural shift. Setting targets as we have revised for as many as 10% of teachers to be rated “Ineffective” and as few as 15% to receive the top rating, both are significant positive departures from today’s “widget” effect, providing the impetus for differentiated development, reward and consequences across the system.
Our revised targets show a narrowing of the gap between high‐ and low‐need schools in terms of the effectiveness of teachers because many of New York’s Race to the Top interventions—such as our teacher compensation and school turnaround initiatives for high‐need schools and districts—are focused on doing just this. The speed and magnitude of that gap narrowing is consistent with our earlier targets, but the absolute numbers are different because our rating categories will represent relative achievement across teachers and principals, which we expect will always be normally distributed. Thus, instead of starting with 25% of teachers rated “Ineffective” and another 25% rated “Highly Effective,” and ending up with 40% “Highly Effective” after 3 years (as in our original targets), we expect the new evaluation system to result in a normal distribution in which 70%‐80% of teachers and principals will fall into either the “Developing” or “Effective” categories each year.
In both high‐ and low‐need districts, as professional development efforts are focused on promising new educators who were rated “Developing,” we would expect to see more dramatic improvements in the percentage of educators who move from the “Developing” into the “Effective” category. This trend is not picked up in the Race to the Top required metrics, but will show up in the supplemental metrics we have suggested.”
This is a plan where the population will be forced to fit the curve and everyone will have about a 50/50 chance of “success” in any given year. In very short order, every teacher and principal in the state will have to have an improvement plan, but won’t be able to show adequate improvement because of the problems associated with standardized testing, SGP model, and VAM, and we’ll all have bad ratings. Talk about Catch-22!
LikeLike
The source of the cancer–as I see it–is the “standardized” (which is hardly standardized, reliable, or valid, according to the half-hearted methodology described in Todd Farley’s excellent 2009 expose, Making the Grades: My Misadventures in the Standardized Testing Industry, a MUST read for anyone who cares about the future of America’s children) testing itself. The entirety of the industry is faulty & incompetent. Teachers are being evaluated–& fired–based on tests having heaven knows how many other ??? like the Pineapple? (As a special ed. teacher/reader, I’ve seen PLENTY more in my last 10 years of teaching!) How many math ??? have either no right answer or 2 or 3 right answers? How many written portions of the tests are incorrectly scored by incompetent or piecemeal-paid (score one essay every 2 minutes) reader? Or–perhaps worse–by COMPUTERS that can’t read? (Read Todd Farley’s latest piece on The Huffington Post.) How many rubrics have been changed by state D.o.Ed. administrators who don’t like the scoring results? How many scores have been erased/changed by scoring supervisors to make the scores APPEAR TO BE “valid & reliable?”
Make no mistake–translate this into the BILLIONS of $$$ being WASTED on testing
which should go to our schools–this is a big money, oligarchical plan that has ABSOLUTELY NOTHING to do with helping students.
In my 55 years of education (35 as a teacher, the remainder as a student), I have NEVER seen the likes of this. It behooves ALL of us–community by community, state by state–to join with locals from PAA (Parents Across America–a national parents’ group which is involved in advocacy through action, & made up of many local affiliates–check out their website {they started in Seattle; their New York group was either partially–or entirely–responsible for the 6/7 Pearson Field Trip; & there’s PURE [Parents United for Responsible Education] in Chicago, where their leader made national news when she outed a ? about charter schools on a test!)–& TAKE ACTION. Parents, Opt Out of testing.
LikeLike
Thanks for the info concerning T. Farley will have to look it up.
For the definitive destruction of educational standards and standardized testing please see N. Wilson’s “Educational Standards and the Problem of Error to be found at: http://epaa.asu.edu/ojs/article/view/577 .
For a shorter essay on the invalidity of standardized testing see Wilson’s “A Little Less than Valid: An Essay Review”.
Click to access v10n5.pdf
I keep referencing these two studies as most have never heard of Wilson. You’ll get to know him when you read the first study cited.
LikeLike
The other problem with the 35% margin of error is that it assumes that you can accurately assess how much value each of us adds to our students based on mere test scores. Gary Rubenstein went into this in his blog series debunking the TDRs. If you count all of the personal, social and unresolved academic issues my students have gone through, and then look at the formula that is used to compensate for these issues, it is clear that the base assumption is flawed.
LikeLike