Now it is teachers in Ohio that have been rated by a secret value-added formula.
Teachers in affluent schools were twice as likely to score well as those in low-income schools.
Here is the key language:
“The details of how the scores are calculated aren’t public. The Ohio Department of Education will pay a North Carolina-based company, SAS Institute Inc., $2.3 million this year to do value-added calculations for teachers and schools. The company has released some information on its value-added model but declined to release key details about how Ohio teachers’ value-added scores are calculated.
“The Education Department doesn’t have a copy of the full model and data rules either.
“The department’s top research official, Matt Cohen, acknowledged that he can’t explain the details of exactly how Ohio’s value-added model works. He said that’s not a problem.”
Think of it. The person at the Ohio Department of Education in charge of the ratings doesn’t understand how the model works. He says it is not a problem.
Well, it is a problem for excellent teachers who were told they were “least effective.”
These models,based on standardized tests, are inaccurate and unstable.
Do not trust the ratings. They are garbage. No high-performing nation is rating teachers this way. It is mean-spirited, mechanistic, and meaningless.
Diane Ravitch's blog 
In 2011, I was ‘above average.’ In 2012, I was ‘average.’ I guess that means I did something different in my teaching? Unfortunately, I don’t know what it was.
And your evaluator doesn’t know either. That is one of the reasons why this is so bogus. Ohio just waxes worse and worse. Our governor just keeps patting his own back because his cohorts want him to “cut government”. He has done so, sending districts, police officers, firefighters, and local city, township, and county governments into tailspins, searching for ways to provide minimum services for everyone. But, hey, do you think voters want less services with less people available to perform them? No. They want to magically be protected and educated and would apparently be glad if volunteers did it. Have a turnover of every 2-3 years in all those areas…you get what you pay for.
Virago, the ratings are inherently unstable and inaccurate. As you have experienced, someone with a high rating one year may get a low rating the next, and vice versa. There is no rhyme or reason.
In 2009 I was average. 2010 bottom of barrel “-3”. 2011 at top with a “+3”. I didnt know what VAM was and only slightly modified my lesson plans.
VAM is junk science.
What. I meant was I barely changed my lesson plans throughout the threee years and VAM was being piloted. I didnt know what it was or how it would be used. I did nothing to receive the top score in 2011.
If I had as much money as Bill Gates, I would fund a sabbatical for every teacher in the country and then let the powers that be with all their almost medieval tortures see what they would do then.
“The company has released some information on its value-added model but declined to release key details about how Ohio teachers’ value-added scores are calculated.”
That’s “proprietary” information and is off limits to all except ASS Inc., oops, I meant SAS Inc.
So Ohio wants its teachers to sign off on an evaluation in which the teachers can’t know how the the evaluation was determined? What is this “double secret probation” from Animal House? Ohio teachers refuse to sign off of that portion of your evaluation or better yet bring a black permanent marker and X off that portion and then sign it.
““The department’s top research official, Matt Cohen, acknowledged that he can’t explain the details of exactly how Ohio’s value-added model works. He said that’s not a problem.”
So, I guess Matt Cohen should be looking for work in a different field as obviously he is unqualified. Who is this guy anyway? I tried looking him up on the Ohio Dept. of Ed. website and couldn’t find anything. Anyone have any link or further information about him? Is he a Broadie?
Duane, so scary–I IMMEDIATELY thought of Animal House’s “double secret probation” upon reading this! I recall that you had given the “black permanent marker/X” solution in yet another case of evaluation malfeasance, and an excellent suggestion it is!
In the “good old days” of teaching–before all this insanity–I had the pleasure of throwing an evaluation back in the face of a snooty assistant principal (she wrote me up for being “unprofessional” at a
special ed. meeting–yeah, I was unprofessional all right– I told the ignorant special ed. director that she needed to have one of our students evaluated or risk violating PL. 94-142 {I had taken both school law and special education law courses for such a reason, & our sp.ed. director hadn’t taken even one, knew zilch, yet insisted–to the parent, no less–that the child would do just fine in the general ed. classroom–not} in front of–gasp!–a parent who didn’t know that her child’s rights were being violated). I had heard the A.P. YELLING at a teacher in the hall in FRONT of her students, just because we had the seniors visiting our school that day, and the A.P. “felt” that the teacher wasn’t watching her kids.
So–I called the A.P. on it–said I’d seen/heard her being “unprofessional,” & would she like it if I reported her behavior to the principal?! Oh, she would NOT, and OF COURSE she would write out the negative part of my evaluation.
Oh, yeah–those were the days!
RBMTK,
You’re right it’s scary. I know I am going to have to take a black permanent marker to my evaluation meeting (with a newbie administrator, unless the principal decides not to subject the newbie to me-ha ha) next year as there is supposed to be a student test score section as part of the evaluation. I know I could “game the system” but I tire of those games quickly and prefer to just do what needs to be done. So be it. It’s hard to be evaluated by someone (and there are no administrators in my building that even begin to coming close to my experience (and experiences outside of teaching) in the classroom. It’s quite hard to listen to folks who have no clue what it takes to learn a second language try to “evaluate” me, actually it’s quite insulting.
Oh, well gotta keep the BP down or else I’ll blow a gasket.
“Teachers in affluent schools were twice as likely to score well as those in low-income schools.”
So assuming the above is true — which it most assuredly is NOT – what would be the correct solution to that problem? Fire the least -effective teachers in those low-income schools and replace them with whom exactly? More brand-new teachers who will be working with some of the most-difficult students in the school system? Sounds like a failed plan to me.
Or are they going to entice the teachers from the more affluent schools to transfer to the low-income schools? Yeah…good luck with that.
Brendan,
“So assuming the above is true — which it most assuredly is NOT. . .”
Why do you say that the statement is not true? What is your thinking/logic behind that statement?
Thanks,
Duane
Sorry – I should have been a little more precise with my language.
From Diane’s statement, I was inferring that if you scored “well” then you were considered “highly-effective” or “effective”, and if you scored “not well” then you were “developing” or “ineffective.”
It’s hard for me to precisely say what the terms in quotes mean to me, as I was using my own understanding of them, not a value-added formula. But I’ll try:
For me, ineffective means you are barely teaching and retaining control of the classroom, while developing means you have a better handle on things, but still need a lot of work.
Effective means that you’re doing a pretty good job but still need to improve things, while highly effective means that you’re doing a damn good job.
Using those definitions of “well” and “not-well,” I think that there are probably similar percentages of those types of teachers in both affluent and low-income schools. However, for the Ohio value-added formula, the results are skewed in favor of the affluent school, for whatever reason. “Whatever reason” because we’re not allowed to see the formula that they used.
Diane wrote: “These models,based on standardized tests, are inaccurate and unstable.” and “Virago, the ratings are inherently unstable and inaccurate. As you have experienced, someone with a high rating one year may get a low rating the next, and vice versa. There is no rhyme or reason.”
One of these days, Diane, I hope you have a similar epiphany in understanding that educational standards and all standardized testing are “inaccurate and unstable”, in other words, invalid, as you did in changing your mind and heart concerning the various other educational practices that you endorsed at one time and now so valiantly fight against. And understand that these practices also harm many as many students as the practices you previously endorsed.
Finally, someone noticed! We’ve had to take it on blind faith that these ratings were correct. They’ve never been able to explain what the formula is that calculates value added. Thank you for pointing it out.
This reminds me of two boys in my seventh grade PE class rating the girls during gym time. They assigned a number based on two criteria (which most people can figure out based on the values of 12 year olds), then presented a notebook paper certificate to their winner at the end, pleased as punch with themselves (at the embarrassing expense of any self-conscious seventh grade girl who was aware what they were doing).
Indeed the era of voting people off the island or sitting back and assigning subjective scores from the safe seat of a judging panel (pop culture) is making its way into reality beyond reality tv.
Thank you for making this public! The entire new teacher eval. system being put in place in Ohio is a totally unorganized and the ODE has shirked their responsibilities.
As a science teacher I resent the fact that I am being held to a standard that many other teachers are not. The band, chorus, P.E., technology, art, family and consumer science teachers, et al do not have the same VAM placed over their heads. The whole idea that our student test scores will gauge us the same way each year is ludicrous as each class has its own culture. So many other factors, socio economic class, attitude, teacher the year or years before, etc. all create a less than static playing field which of course will make scores vary. I think it is an effort by many to try to make the complex nature of teaching and learning understandable. We need to start to utilize testing as a diagnostic tool, learn what a student needs to improve; rather than as an autopsy to learn what “killed” them.
Bob, you are right. VAM is junk science. There are many unmeasured variables.
Bob: you should think collectively at this point, “How can WE TEACHERS change this absurd system together”. Saying “Why me and not them” is futile and makes a bad situation worse. I agree that you are being rated on nonsense! Also, if you were “THEM” (the PE, music, art teachers) you would be having to be rated on test scores that are not applicable to what you teach and you may even have scores of students factor into your VAm and you never even taught them! There are lawsuits in Florida over this! All teachers are effected by this so we need to figure out how to work together!
With all respect to the many educators that post on this blog: name a typical blue- or white- collar job where your evaluation—with possible termination for severely unsatisfactory performance—was based on a secret formula?
Couple that with the VIP in charge of understanding and explaining the rating system doesn’t think it’s a problem that he’s as much in the dark as you.
Franz Kafka just got a beat down. All he could come up with was “The Trial” and “The Metamorphosis.” Looks like Ohio is better at, well, Mark Twain said it best:
“Truth is stranger than fiction, but it is because Fiction is obliged to stick to possibilities; Truth isn’t.”
And Karen Lewis is an extremist???
🙂
I haven’t read much of Charlotte Danielson’s work, but I recall picking up her making the point that it is important that teachers being evaluated be told the evaluation standards; because otherwise, they won’t be able to improve.
Aside from being the advice of a leading teacher evaluation reform movement guru, the importance of informing an employee of evaluation standards is surely Human Resources Management 101.
An important reason for having clear procedures and standards regarding of public school teachers (as opposed to private school teachers) is that public school systems are subjected to political pressures, and can be corrupted. One would think that applying a secret formula to evaluate teachers violates their rights as government employees. A lack of transparency in how teachers are evaluated is not fair to parents, students, and non-parent taxpayers either, because it means that they cannot judge how well the school system is working.
All in all, the Ohio system sounds like a bad idea for everyone — except people who may have an interest in firing teachers, while giving reasons that sound reasonable and can’t reviewed by anyone who might find them lacking.
It is the most open-ended idea there could be. There is no closed loop, no feedback. The question is how can someone ever succeed? This is going to create such poor morale in the schools that those who it is meant to help will suffer; the children.
“Human Resources Management 101.” AAAAAARRRRGGGGHHHH!
I hate that term. I remember back in the 80s when that started to replace “personnel”. Pure exploitation language. Eff those who continue promoting/using that concept. Disgustingly abhorrent use of other people.
Thank Krazy TA, I wonder, if this is brought out in court, will the court have to protect the NC company’s proprietary rights and be sequestered? Kafka may yet be vindicated by reality. Perhaps he wasn’t writing fiction, but, like Orwell, just got the time off a little bit.
What kind of crazy game are they doing there? The state is going to use a methodology that they do not even understand. How is this reliable? How can you control peoples lives and futures with an unknown formula? How is this acceptable and legal?
I teach in a small district in Ohio and today our students’ test scores were released. With sweaty palms and arrhythmic heartbeats, many of us typed in our information to gain access into the testing site. What followed was typical of many of us; shrieks of, “How??? Why??? What now???” as we poured over student data. Did we work any less than last year when our overall scores were higher? No! Did we quit trying various ways to engage, differentiate and reach each student in our classrooms? No! Did we simply try “one approach fits all” methods in our classrooms this year? Of course not! So, what happened? What changed? Why the slight decline in this year’s scores compared to the previous year’s scores? Well, the answer is as obvious as the nose on our governor’s smirking face…… Our STUDENTS change each year along with that the socioeconomic status of their parents and the amount of involvement that they are willing or able to devote to their child ‘s education. To evaluate Ohio teachers based on a system that does not include this important aspect of learning is akin to evaluating a governor’s ability to effectively run a state based on his own economic status and education and not his constituents. Of course we teachers want the best for our students and we will continue to employ any and all possible methods to ensure their success, but can we work together with state leaders, administers and parents to create a system where educators are not singled out as scapegoats for a state’s woes, but are helped to do their best for those children in each classroom instead of being hindered, harangued and habitually attacked? Perhaps then we will be able to prepare our young for a global economy and teach students the value and importance of their role within that economy.
Jemmi Boso: thank you for working so hard even when they’re sucker punching you.
Just remember that the self-styled “education reformers” think you’ve got it coming. After all, in their world of creative destruction and cagebusting twenty first century educational excellence, they get all the credit and $tudent $ucce$$ while you get all the pain and blame and pink slips.
“By trying we can easily endure adversity. Another man’s, I mean.” [Mark Twain]
Don’t agonize. Organize.
🙂
Ditto everything in this article for Florida. I got a “Highly Effective” rating for my Administrator’s evaluation and I’m now awaiting the VAM for my students test scores from our standardized state test which have nothing to do with my subject area. Somehow, that will be calculated with my evaluation score to determine what kind of teacher I am and how much I will be paid. These are real geniuses we’re dealing with here. I will not sign off on this ridiculousness either.
If you think that teachers art, music, and other “non-tested” subjects are off the hook in Ohio, you have not seen the boilerplate Student Learning Objectives (SLOs) with 26 criteria for writing them “properly” or how teachers are graded on this version of WW II “how to write a training objective for a gunner,” including the task of setting your “targets” for learning–a projection of the percentage of students whose scores on pre-tests will be increased by the end of the grade or course, and what cut scores you will set for the post-test, or the rule that says you CANNOT change this expected productivity measure after someone (unknown credentials in your subject) gives you a grade on this writing and research assignment that makes all sorts of ridiculous assumptions about your job assignment available research testing instruments in your field, and so on. You probably have not looked at the instructions for gathering your “data” by mid April for the convenience of evaluators who will report before May 1, on your total score, 50% from whether you “hit your SLO targets.” Fifty percent of your evaluation as a teacher is based on meeting your immovable targets, and frozen-in-time curriculum tailored to meet them. IF you think the “other 70%” of teachers who do not have SAS VAM scores have an easy time, you have not looked at the bizarre instructions for you to enter your posttest scores in an Excel spreadsheet, enabling an UNDISCLOSED formula to compute your value-added score, and that is the bottom line… because the back-up data on what your students have learned, portfolios, perhaps audio and video records of performances, and the rest does not really count. You probably have not seen that these requirements explicitly preclude any teacher made rubrics and tests for their own classes, a by-product of a 2010 federal regulation that says measures of student learning must be comparable across classrooms (perhaps school, more likely at the district level). Any teacher who dares to have a distinctive program with multidisciplinary activites and some team teaching, and fortuitous activities, or objectives discerned AFTER setting those SLO “targets” is headed for trouble. Oh yes. The students who score low on the pretest will have to more, at a faster rate that the kids who score high. Learn more faster. That is literally what Race to the Top means, and Ohio is on board for that. Don’t get me started on how the OTHER 50% of this evaluation scheme works for teachers of “non-tested” subjects.
I’ve been a classroom teacher in Ohio for 23 years, and I’ve weathered my share of mandates from on high. There’s something quite different about this latest revolution, though: instead of focusing on what and how students learn, the new Ohio Teacher Evaluation System (OTES) is about blaming teachers, mostly for factors beyond our control. I sent the following email to my principal the weekend before I was scheduled to be evaluated under this new rubric:
“I mean absolutely no disrespect to you or any other administrator by this email, but I have decided to opt out of the pre-observation conference. I have submitted the pre-assessment profile and the self-assessment via PD360, and I will provide you with a lesson plan before the scheduled date of my observation. On the day of your visit, however, my students will be reading silently for the entire period.
I’ve jumped through every hoop (eleven hours on one lesson), but I cannot, in good conscience, take part in a ranking system designed by people who do not respect my profession. What if I designed an assessment for my students and told them, “None of you will get an A. Most of you will get Cs and Ds. I’m doing this to make you better”? This entire system is toxic. It’s attempting to do what Senate Bill 5 couldn’t: divide us. I refuse to enter into a competition with my colleagues. How can I sign onto that and still look my students in the eye?
I hope you’ll understand my decision.”
Needless to say, the email landed me in all varieties of hot water in my district. I am currently rated “ineffective”–(I was District Teacher of the Year in 2009)–and have been placed on an “improvement plan.”
My argument, of course, is not with being held accountable; I’ve been accountable my entire career. My argument is with a system (designed by legislators in the state who’ve already proven their distaste for public education by refusing to fund it properly) that points the finger at teachers by attempting to quantify something as slippery and wonderful as the ability to inspire.
This entire “school reform” movement isn’t making us better; it’s making us bitter.