I earlier reported that the latest data show that 97% of teachers in Pittsburgh received ratings of either “distinguished” or “advanced.” Similar findings have emerged elsewhere, which makes me wonder why it was necessary to spend billions of dollars to create these new evaluation systems, which are often incomprehensible. But Kipp Dawson, a Pittsburgh teacher wrote a comment warning that the evaluation system is flawed and riddled with unreliable elements, like VAM. Don’t be fooled, Dawson says. The Pittsburgh evaluation system was created with the lure of Gates money. It attempts to quantify the unmeasurable.
Dawson writes:
I am a Pittsburgh teacher and an activist in the Pittsburgh Federation of Teachers (AFT). Let’s not let ourselves get pulled into the trap of applauding the results of a wholly flawed system. OK, so this round the numbers look better than the “reformers” thought they would. BUT the “multiple measures” on which they are based are bogus. And it was a trap, not a step forward, that our union let ourselves get pulled in (via Gates money) to becoming apologists for an “evaluation” system made up of elements which this column has helped to expose as NOT ok for “evaluating” teachers, and deciding which of us is an “effective” teacher, and which of us should have jobs and who should be terminated.
A reminder. VAM. A major one of these “multiple measures.” Now widely rejected as an “evaluating” tool by professionals in the field, and by the AFT. A major part of this “evaluation” system.
Danielson rubrics, another major one of these multiple measures: after many permutations and reincarnations in Pittsburgh, turned into the opposite of what they were in the beginning of this process — presented to us as a tool to help teachers get a window on our practice, but now a set of numbers to which our practice boils down, and which is used to judge and label us. And “objective?” In today’s world, where administrators have to justify their “findings” in a system which relies so heavily on test scores? What do you think . . .
Then there’s (in Pittsburgh) Tripod, the third big measure, where students from the ages of 5 (yes, really) through high school “rate” their teachers — which could be useful to us for insight but, really, a way to decide who is and who is not an “effective” teacher?
To say nothing of the fact that many teachers teach subjects and/or students which can’t be boiled down in these ways, so they are “evaluated” on the basis of other people’s “scores” over which they have even less control.
Really, now.
So, yes, these numbers look better than they did last year, in a “practice run.” But is this whole thing ok? Should we be celebrating that we found the answer to figuring out who is and who is not an “effective” teacher?
This is a trap. Let’s not fall into it.
Diane Ravitch's blog 
Latest from the NEA. http://www.nea.org/home/we-wont-give-up.html
LikeLike
Lost in all of this is one truth, it is primarily the responsibility of the student and their family to take advantage of the FREE education that is being offered to them. NCLB abrogated that responsibility.
Nothing will change until that is a cultural given.
A corollary to that: if you want to help kids, help their parents and guardians.This means get them better working conditions and stop treating workers like garbage. Whoops, that means changing the class power structure.
It’s all so much easier to treat teachers like garbage.
LikeLike
http://www.brookings.edu/blogs/up-front/posts/2014/06/13-fighting-poverty-national-policy-priority-kearney-harris
LikeLike
http://www.brookings.edu/blogs/social-mobility-memos/posts/2014/05/22-chronic-stress-hurts-brain-development-social-mobility-reeves-howard
LikeLike
Reformers are all barking up the wrong tree. In fact their entire premise is wrong. The weakest link in the learning/achievement chain is rarely the teacher.
I have one hundred students this year. I teach a subject that is new to all of them. This puts all of my 14 year old students on an equal footing as far as course content goes. After a year of instruction, a few of them have been incredibly successful. Some have done quite well. Most have done ok, just not setting the academic world on fire, And a few have been abysmal failures. If student achievement rests solely (or even mostly) on the teacher, how can this be explained?
The premise behind the teacher bashing movement is based on the surrealistic notion that virtually all students are willing participants in the learning process. If all my students attend school daily and are eager to learn, attentive, inquisitive, organized, conscientious, and hard working (and had the necessary parental support), I will gladly take the blame for student failure (In fact I will file the 3020A myself). Now in school districts where this is largely the case, we falsely praise the teachers for being so highly effective. Every teacher with smart and motivated students looks like a genius; some of them even believe their own press clippings. It is my strong belief that teachers should not be required to be magical, amazing superstars in order for students to simply do their job.
We are looking for reasons why students don’t learn, and the only rock we have not looked under is the only rock worth exploring.
LikeLike
The reformers don’t give a dingleberry how students are doing – its all a, now thinly veiled, disguise to get rid of unions, get rid of the teaching profession, get rid of public schools, open private-but-publicly-funded non-stop gravy-train of dollars of profits, then, convert the charters into online learning with minimum cash outlay on “product” to deliver to the CUSTOMER (student). As they say in the theatre “and, scene.” Done. Finished.
Who is going to hold the customers accountable when they are sat in front of computer screens? Perhaps that is the need for the biofeedback, face recognition, chairs that feel you squirm, electronic bracelet tethers, etc. to “judge” customer/student interest in what is being “taught.”
If we could look 20 years into the future…the above doesn’t sound so wrong does it? Perhaps the customers’ parents will get arrested for not having the customer show up to the computer screen? Have y’all seen this: http://www.huffingtonpost.com/2014/06/11/eileen-dinino-dead-pennsylvania-jail_n_5486353.html
Wave of the future. Kids, not in school, but in front of a computer screen, and the parents jailed when Johnny can’t learn. Oh, the sheer evil of it all. Gotta give it to the Waltons, Broad, Gates, Koch, Rhee.
LikeLike
This would be a good time for the teachers of Pittsburgh to hold a public meeting for the purpose of burning those evaluations. What a contemptible waste of resources this entire evaluation procedure is when there are kids in need and, yes, buying into it simply encourages, further, such heedless squandering of public energy, attention, and resources.
LikeLike
Here’s what that looks like in New Mexico. http://krqe.com/2014/05/26/taos-teachers-burn-their-evaluations/
LikeLike
Kipp Dawson is 100% correct. Shame on the AFT for applauding this treacherous and duplicitous attach on teachers. Pittsburgh is under the same Gates funded tyranny as Hillsborough County Florida. Danielson plus VAM does not a fair evaluation system system make – no matter how many teachers are rated effective. Keep in mind that where they place the bar is TOTALLY arbitrary; there is no magic number in their voodoo math that represents the true cutoff between a good and bad teacher. Next year, they can place the bar wherever they wish.
The Gates bribery of unions is about firing teachers; make no mistake about that. The agreement with Hillsborough County calls for the dismissal of 5% of “tenured” teachers each year and the denial of due process to a great many more. Just because they have yet to lower the boom is no reason to applaud them.
Three words: Don’t be stupid!
LikeLike
Also, when the numbers from the Danielson rubric are too high, they simply “recalibrate.” This term is used to mislead the masses into believing that the district somehow scientifically adjusts objective measurements, when, in reality, they simply demand that evaluators subjectively give fewer Accomplished and Exemplary ratings. Pittsburgh will surely be hearing this magic word in the near future.
LikeLike
They are trying to norm reference the teacher evaluation process. Idiots! Had the same experience in my district. They wanted fewer highly effective teachers so they arbitrarily changed the SLO growth bands to make it more difficult to show student growth. Administrators drinking this Kool Aide undermine their credibility and piss off the very people (veteran teachers) they should be asking for help and advice.
LikeLike
Ha Ha – Thank you for reminding me of “recalibration”! I did a presentation to the Hillsborough County School Board, one evening when they were approving a contract to have “specialists” come in and “recalibrate” the evaluators. I proved (using real math, not voodoo sham deformer math), that the multiplicative error in recalibration (even if you made the huge leap of giving it some subjective reality) resulted in a 100% difference between recalibrators and recalibratees being acceptable.
No interest whatsoever in facts and real mathematics.
LikeLike
Have you ever watched danielson videos of so called highly effective teachers, etc. They are a joke. Who determined that one way of teaching was superior? I’ve seen videos of teachers walking around the room while students are in groups. They may ask some questions and move on. My own children say that group work is more social time. It is all a joke. An evaluator can write anything they want on an evaluation to get rid of someone.
LikeLike
Such videos seem like SNL skits to good, veteran teachers.
LikeLike
Kipp is, as usual, right on the money. We need to take all good news with heavy doses of salt and be on the lookout for traps and ulterior motives. Gates Foundations’s recent call for a moratorium on testing—the very testing in the VAM used by Pittsburgh Schools—comes to mind. Give the deformers credit for looking a few steps down the road, and spinning events in ways that will further their cause.
LikeLike
“It attempts to quantify the unmeasurable.”
BINGLE, BANGLE, BOINGLE!!!!
Until teachers, administrators, parents and the community reject such ILLOGICAL, INVALID AND UNETHICAL methods of assessing the teaching and learning process the carnage and destruction of that process will continue. Those who have promoted and/or willingly and enthusiastically “gone along to get along” (GAGAers) should be shunned and banned for life from the teaching profession for these educational malpractices that have harmed many INNOCENT children.
LikeLike
I still don’t understand how Gates and his sycophants really believe they are doing anything to improve education. They are merely hurting teachers.
LikeLike
They DON’T believe they are improving education, or student outcomes, or effective teaching. Remember, this is about dismantling public education so it can continue to be privatized, but funded with taxpayer dollars. The ultimate goal, despite the rhetorical lip service, is that k-12 schools and teaching as we currently know it, disappear.
LikeLike
“It attempts to quantify the unmeasurable.”
BINGLE, BANGLE, BOINGLE!!!!
To understand why these educational malpractices (educational standards-oh but we must have standards to bow down before-and standardized testing-the oh so great ‘objective’ measurement of students’ ‘achievement’ (sic)) one only needs to read and understand Noel Wilson’s never refuted not rebutted seminal work “Educational Standards and the Problem of Error” found at: http://epaa.asu.edu/ojs/article/view/577/700
Brief outline of Wilson’s “Educational Standards and the Problem of Error” and some comments of mine. (updated 6/24/13 per Wilson email)
1. A quality cannot be quantified. Quantity is a sub-category of quality. It is illogical to judge/assess a whole category by only a part (sub-category) of the whole. The assessment is, by definition, lacking in the sense that “assessments are always of multidimensional qualities. To quantify them as one dimensional quantities (numbers or grades) is to perpetuate a fundamental logical error” (per Wilson). The teaching and learning process falls in the logical realm of aesthetics/qualities of human interactions. In attempting to quantify educational standards and standardized testing we are lacking much information about said interactions.
2. A major epistemological mistake is that we attach, with great importance, the “score” of the student, not only onto the student but also, by extension, the teacher, school and district. Any description of a testing event is only a description of an interaction, that of the student and the testing device at a given time and place. The only correct logical thing that we can attempt to do is to describe that interaction (how accurately or not is a whole other story). That description cannot, by logical thought, be “assigned/attached” to the student as it cannot be a description of the student but the interaction. And this error is probably one of the most egregious “errors” that occur with standardized testing (and even the “grading” of students by a teacher).
3. Wilson identifies four “frames of reference” each with distinct assumptions (epistemological basis) about the assessment process from which the “assessor” views the interactions of the teaching and learning process: the Judge (think college professor who “knows” the students capabilities and grades them accordingly), the General Frame-think standardized testing that claims to have a “scientific” basis, the Specific Frame-think of learning by objective like computer based learning, getting a correct answer before moving on to the next screen, and the Responsive Frame-think of an apprenticeship in a trade or a medical residency program where the learner interacts with the “teacher” with constant feedback. Each category has its own sources of error and more error in the process is caused when the assessor confuses and conflates the categories.
4. Wilson elucidates the notion of “error”: “Error is predicated on a notion of perfection; to allocate error is to imply what is without error; to know error it is necessary to determine what is true. And what is true is determined by what we define as true, theoretically by the assumptions of our epistemology, practically by the events and non-events, the discourses and silences, the world of surfaces and their interactions and interpretations; in short, the practices that permeate the field. . . Error is the uncertainty dimension of the statement; error is the band within which chaos reigns, in which anything can happen. Error comprises all of those eventful circumstances which make the assessment statement less than perfectly precise, the measure less than perfectly accurate, the rank order less than perfectly stable, the standard and its measurement less than absolute, and the communication of its truth less than impeccable.”
In other word all the logical errors involved in the process render any conclusions invalid.
5. The test makers/psychometricians, through all sorts of mathematical machinations attempt to “prove” that these tests (based on standards) are valid-errorless or supposedly at least with minimal error [they aren’t]. Wilson turns the concept of validity on its head and focuses on just how invalid the machinations and the test and results are. He is an advocate for the test taker not the test maker. In doing so he identifies thirteen sources of “error”, any one of which renders the test making/giving/disseminating of results invalid. As a basic logical premise is that once something is shown to be invalid it is just that, invalid, and no amount of “fudging” by the psychometricians/test makers can alleviate that invalidity.
6. Having shown the invalidity, and therefore the unreliability, of the whole process Wilson concludes, rightly so, that any result/information gleaned from the process is “vain and illusory”. In other words start with an invalidity, end with an invalidity (except by sheer chance every once in a while, like a blind and anosmic squirrel who finds the occasional acorn, a result may be “true”) or to put in more mundane terms crap in-crap out.
7. And so what does this all mean? I’ll let Wilson have the second to last word: “So what does a test measure in our world? It measures what the person with the power to pay for the test says it measures. And the person who sets the test will name the test what the person who pays for the test wants the test to be named.”
In other words it measures “’something’ and we can specify some of the ‘errors’ in that ‘something’ but still don’t know [precisely] what the ‘something’ is.” The whole process harms many students as the social rewards for some are not available to others who “don’t make the grade (sic)” Should American public education have the function of sorting and separating students so that some may receive greater benefits than others, especially considering that the sorting and separating devices, educational standards and standardized testing, are so flawed not only in concept but in execution?
My answer is NO!!!!!
One final note with Wilson channeling Foucault and his concept of subjectivization:
“So the mark [grade/test score] becomes part of the story about yourself and with sufficient repetitions becomes true: true because those who know, those in authority, say it is true; true because the society in which you live legitimates this authority; true because your cultural habitus makes it difficult for you to perceive, conceive and integrate those aspects of your experience that contradict the story; true because in acting out your story, which now includes the mark and its meaning, the social truth that created it is confirmed; true because if your mark is high you are consistently rewarded, so that your voice becomes a voice of authority in the power-knowledge discourses that reproduce the structure that helped to produce you; true because if your mark is low your voice becomes muted and confirms your lower position in the social hierarchy; true finally because that success or failure confirms that mark that implicitly predicted the now self evident consequences. And so the circle is complete.”
In other words students “internalize” what those “marks” (grades/test scores) mean, and since the vast majority of the students have not developed the mental skills to counteract what the “authorities” say, they accept as “natural and normal” that “story/description” of them. Although paradoxical in a sense, the “I’m an “A” student” is almost as harmful as “I’m an ‘F’ student” in hindering students becoming independent, critical and free thinkers. And having independent, critical and free thinkers is a threat to the current socio-economic structure of society.
LikeLike
Another easy (and wrong) answer to an incredibly complex question. As long as it helps those in power get what they want, the charade parade will move forth.
LikeLike
The question (student achievement/learning) is not that complex, nor is the answer. Given the extraordinary diverse population in America’s schools, given the fantastic range of out of school experiences, given the levels of family dysfunction, and given the wide ranging gene pool, given the extraordinary range of cultural influences, and given the very narrow pathway for success, why would we expect anything different than we do?
LikeLike
Pittsburgh’s evaluation system sounds almost identical to the one we have in Memphis, another district inundated with Gates money.
LikeLike
Hardly a coincidence, is it. But, then, the “example” has been picked up by like-minded (similarly funded, or not, as the case may be) “reformers” across the country. And now it’s going international . . .
LikeLike
It’s similar to Utah’s as well, and we don’t even GET Gates’ money!
LikeLike
Dawson’s point is crucial. It’s a fatal mistake to support or accept a flawed system just because it happens to be going your way at the moment. In fact, the moment it’s going your way is the perfect time to speak against it, because you are clearly not filled with sour grapes.
I doubt that reformsters are looking at Pittsburgh results and saying, “Well, look! Pittsburgh teachers are absolutely awesome! Problem solved!” Much more likely that they’re saying, “Well, that can’t be right. We’d better tinker with this system until it gives us the results we’re looking for.”
LikeLike
Reformers are salivating and getting ready to drop the boom.
LikeLike
“tinker”?
You are being uncharacteristically gentle, no?
LikeLike
Peter, I shared this comment as part of the discussions going on on Facebook, and it is being widely “liked” and shared. Because it’s spot on (“tinker” or not).
LikeLike
Meaning your comment, Peter. (sorry; not clear before)
LikeLike
Excellent insight.
LikeLike