Celia Oyler is a teacher educator at Teachers College, Columbia University.
In this post, she explains that the Chancellor of the Néw York Board of Regents, Merryl Tisch, does not understand how the teacher evaluation plan she snd 10 other Regents just approved works.
Tisch thinks she solved the problem of VAM mistakes by permitting teachers like Sheri Lederman to appeal ratings that are clearly wrong.
Oyler says that Sheri Lederman’s rating, egregiously wrong, was not an “aberration.” The whole system is flawed.
“What is extremely important for all New York State educators and families to understand is that the Chancellor of the Board of Regents does not understand a very basic aspect of a policy she has foisted upon us.”
Diane Ravitch's blog 
Really ? It’s a Flawed system you say, today, in 2015?
IN 1998 this happened to me, and over the next sixteen years, while the media spun lies, and the unions looked the other way, THIS PROCESS took out our nation’s veteran teachers.
http://www.perdaily.com/2011/01/lausd-et-al-a-national-scandal-of-enormous-proportions-by-susan-lee-schwartz-part-1.html
http://www.opednews.com/Quicklink/FINALLY-TARGETED-TEACHERS-in-Best_Web_OpEds-Agenda_America_Corporate_Corruption-150708-830.html#comment553842
NO flaw, but a conspiracy to corrupt the administration of our schools, in order to erase the civil rights of the classroom practitioners who know WHAT LEARNING ACTUALLY LOOKS LIKE, and to replace it with the magic elixirs of Broad, Gates, Pearson and the oligarchs who are eradicating THE INSTITUTION of public education…. and if you wonder how this FLAW, came about take a look at http://citywatchla.com/8box-left/6666-lausd-and-utla-complicity-kills-collective-bargaining-and-civil-rights-for-la-s-teachers
http://www.opednews.com/articles/Magic-Elixir-No-Evidence-by-Susan-Lee-Schwartz-130312-433.html
This is a planned conspiracy to end our input into the practice of this profession, and to make the schools a business where employees can be judged by subjective criteria whites nothing whatsoever to do with their talent, education or performance…or with the OBJECTIVES FOR LEARNING
http://www.opednews.com/articles/Learning-not-Teacher-evalu-by-Susan-Lee-Schwartz-111001-956.html
Much easier to list what Meryl Tisch knows: {}
{ø}
The empty set.
Love it !😎
Thank you, Celia Oyler.
Perhaps Tisch should consider creating an “appeals process” for students who have been subject to bullying.
“Perhaps Tisch should consider creating an “appeals process” for TEACHERS who have been subject to bullying.
I recall the SED attorney calling Sheri Lederman’s data points an outlier, ie out of the normal range, but calling her an “aberration” makes her sound like a freak of nature. She is, to my knowledge, an exemplary teacher who’s confused by how some flawed statistical model can call her anything but excellent. New York teachers need to challenge this VAM formula as it gives the governor power to end careers. Thank you to Sheri and her husband for stepping forward to start the challenge.
They call her an “aberration” and “outlier” because they want people to believe that her case is extraordinary.
In other words, ‘it doesn’t happen very often and hence, there is nothing to be concerned about.’
But the truth of the matter is that VAM scores can (and do) swing significantly from year to year and from model to model. And it is not at all unusual that a teacher rated “effective’ or even “highly effective” one year is rated lower the very next (or vice versa)
See Gary Rubinstein’s series of posts on NY City VAM scores
And the large standard error associated with the VAM scores (large relative to the score difference between what are labeled as ‘effective’ and what might be classed as “less than effective’ teachers) makes even the classification in any given year highly questionable. see this article, for example
What Tisch, Cuomo and the NY state lawyers don’t want the public to realize is that Sheri Lederman is the tip of the proverbial iceberg, most of which is hidden below the water’s surface, waiting to wreck any ship (or teacher, in this case) that happens to hit it.
Well, I hadn’t heard this, however a relevant point is, “how many ‘outliers’ (and, please be specific in your definition) remain so for, say, three years in a row? And, of course, the big question is, “What is the measure actually measuring, and can it be shown that this is important to the quality of education being received (with statistical certainty of, say, 0.1 percent since it will destroy careers, of course)?
The real formula is a big secret, even though it has a 50% weight in teacher’s evaluation. We don’t know the specifics, although some aspects of the VAM system were mentioned in the Lederman case. For more specifics, you can go back and find the posts relating to the case.
My view has been and is now
The reform movement has never focused on the basic philosophy of education. They have “majored on the minors”, lost focus on what the very basic, most fundamental issues of what people education is about.
My letter to our state representative began by stating
Define education. The term is tossed around but never defined with precision. If we don’t know what our goals are how do we know when we have arrived.
Indeed!
amen
Ms Tisch is a prime example of someone who obtains power not based on their knowledge or experience, but rather through wealth. The ramifications of a scenario such as the one that she is the center of is harmful to New York’s public education system and the students within it at a minimum, and may in fact prove to be the destruction of our academia, our public schools, our teachers, and the lives, hopes, dreams, hopes, and aspirations of our children.
Five hundred years ago, the Catholic Church experienced corruption in the form of wealthy individuals using their powers to purchase positions as priests…it was one of the reasons that contributed to the Reformation and splitting of the Christian Church.
The power that Ms Tisch holds is immense, but it is dangerously in the hands of a person highly unqualified to be wielding it.
Ms Tish must be forced to resign immediately, or forcibly be removed from office.
For the sake of our children…for the sake of their future…for our sake as citizens of the state of New York.
Problem is that most people think a number is a precise measurement. I find it forever irritation that a ‘margin of error’ is almost always missing. This is how some are allowed to lie with statistics.
Gauss was a great man. Too bad so few understand what he did by producing a measurement of inaccuracy. A number should ALWAYS be accompanied with that measurement.
Gauss would be agaussed at what is being done in his name
Sheri and I were angry at hearing Sheri’s situation described as an outlier and aberration by both the Attorney General in Court and Meryl Tisch on the Radio. This post properly points out that the issue is that the NY Growth Model does not work properly.
Equally important but overlooked in discussion of the new “appeal process” is that it is only available in limited situations. I read the regulations and they seem designed to protect SED against embarrassing claims like those of Sheri where a district finds a teacher highly effective but VAM finds the teacher ineffective. As I read it, no appeal is allowed for an effective teacher who gets a developing VAM rating. If appeals are allowed, they should be allowed to anyone who thinks VAM has rated them unfairly.
“This post properly points out that the issue is that the NY Growth Model does not work properly.”
And it can never work properly because the “Growth Model” is based on the complete invalidities that are standardized test scores. Start with an invalidity and one, by definition, has to end with invalidity.
Noel Wilson has proven the epistemological and ontological errors and falsehoods and psychometric fudges in the educational standards and standardized testing regime that render any conclusions COMPLETELY INVALID. To understand why read and comprehend his never refuted nor rebutted “Educational Standards and the Problem of Error” found at: http://epaa.asu.edu/ojs/article/view/577/700
Brief outline of Wilson’s “Educational Standards and the Problem of Error” and some comments of mine. (updated 6/24/13 per Wilson email)
1. A description of a quality can only be partially quantified. Quantity is almost always a very small aspect of quality. It is illogical to judge/assess a whole category only by a part of the whole. The assessment is, by definition, lacking in the sense that “assessments are always of multidimensional qualities. To quantify them as unidimensional quantities (numbers or grades) is to perpetuate a fundamental logical error” (per Wilson). The teaching and learning process falls in the logical realm of aesthetics/qualities of human interactions. In attempting to quantify educational standards and standardized testing the descriptive information about said interactions is inadequate, insufficient and inferior to the point of invalidity and unacceptability.
2. A major epistemological mistake is that we attach, with great importance, the “score” of the student, not only onto the student but also, by extension, the teacher, school and district. Any description of a testing event is only a description of an interaction, that of the student and the testing device at a given time and place. The only correct logical thing that we can attempt to do is to describe that interaction (how accurately or not is a whole other story). That description cannot, by logical thought, be “assigned/attached” to the student as it cannot be a description of the student but the interaction. And this error is probably one of the most egregious “errors” that occur with standardized testing (and even the “grading” of students by a teacher).
3. Wilson identifies four “frames of reference” each with distinct assumptions (epistemological basis) about the assessment process from which the “assessor” views the interactions of the teaching and learning process: the Judge (think college professor who “knows” the students capabilities and grades them accordingly), the General Frame-think standardized testing that claims to have a “scientific” basis, the Specific Frame-think of learning by objective like computer based learning, getting a correct answer before moving on to the next screen, and the Responsive Frame-think of an apprenticeship in a trade or a medical residency program where the learner interacts with the “teacher” with constant feedback. Each category has its own sources of error and more error in the process is caused when the assessor confuses and conflates the categories.
4. Wilson elucidates the notion of “error”: “Error is predicated on a notion of perfection; to allocate error is to imply what is without error; to know error it is necessary to determine what is true. And what is true is determined by what we define as true, theoretically by the assumptions of our epistemology, practically by the events and non-events, the discourses and silences, the world of surfaces and their interactions and interpretations; in short, the practices that permeate the field. . . Error is the uncertainty dimension of the statement; error is the band within which chaos reigns, in which anything can happen. Error comprises all of those eventful circumstances which make the assessment statement less than perfectly precise, the measure less than perfectly accurate, the rank order less than perfectly stable, the standard and its measurement less than absolute, and the communication of its truth less than impeccable.”
In other words all the logical errors involved in the process render any conclusions invalid.
5. The test makers/psychometricians, through all sorts of mathematical machinations attempt to “prove” that these tests (based on standards) are valid-errorless or supposedly at least with minimal error [they aren’t]. Wilson turns the concept of validity on its head and focuses on just how invalid the machinations and the test and results are. He is an advocate for the test taker not the test maker. In doing so he identifies thirteen sources of “error”, any one of which renders the test making/giving/disseminating of results invalid. And a basic logical premise is that once something is shown to be invalid it is just that, invalid, and no amount of “fudging” by the psychometricians/test makers can alleviate that invalidity.
6. Having shown the invalidity, and therefore the unreliability, of the whole process Wilson concludes, rightly so, that any result/information gleaned from the process is “vain and illusory”. In other words start with an invalidity, end with an invalidity (except by sheer chance every once in a while, like a blind and anosmic squirrel who finds the occasional acorn, a result may be “true”) or to put in more mundane terms crap in-crap out.
7. And so what does this all mean? I’ll let Wilson have the second to last word: “So what does a test measure in our world? It measures what the person with the power to pay for the test says it measures. And the person who sets the test will name the test what the person who pays for the test wants the test to be named.”
In other words it attempts to measure “’something’ and we can specify some of the ‘errors’ in that ‘something’ but still don’t know [precisely] what the ‘something’ is.” The whole process harms many students as the social rewards for some are not available to others who “don’t make the grade (sic)” Should American public education have the function of sorting and separating students so that some may receive greater benefits than others, especially considering that the sorting and separating devices, educational standards and standardized testing, are so flawed not only in concept but in execution?
My answer is NO!!!!!
One final note with Wilson channeling Foucault and his concept of subjectivization:
“So the mark [grade/test score] becomes part of the story about yourself and with sufficient repetitions becomes true: true because those who know, those in authority, say it is true; true because the society in which you live legitimates this authority; true because your cultural habitus makes it difficult for you to perceive, conceive and integrate those aspects of your experience that contradict the story; true because in acting out your story, which now includes the mark and its meaning, the social truth that created it is confirmed; true because if your mark is high you are consistently rewarded, so that your voice becomes a voice of authority in the power-knowledge discourses that reproduce the structure that helped to produce you; true because if your mark is low your voice becomes muted and confirms your lower position in the social hierarchy; true finally because that success or failure confirms that mark that implicitly predicted the now self-evident consequences. And so the circle is complete.”
In other words students “internalize” what those “marks” (grades/test scores) mean, and since the vast majority of the students have not developed the mental skills to counteract what the “authorities” say, they accept as “natural and normal” that “story/description” of them. Although paradoxical in a sense, the “I’m an “A” student” is almost as harmful as “I’m an ‘F’ student” in hindering students becoming independent, critical and free thinkers. And having independent, critical and free thinkers is a threat to the current socio-economic structure of society.
Duane,
Why not quote Wilson in this instance?
🙂
You’ve got me confundido, Robert unless my sarcasmometer is broken. But to answer the question of “Why quote Wilson in this instance?
A couple of reasons. First to show that the basis for VAM, i.e., using standardized test scores is invalid. And if the underlying/underpinning fundamental structure isn’t good, valid, epistemologically and ontologically solid then any conclusions by definition have to be invalid, false, or as Wilson puts it “vain and illusory”.
Second and third, because I haven’t posted it in a little bit and I am attempting to prove Oye’s corollary to Goebbel’s, Stalin’s thesis of telling a big enough lie often enough it becomes true, which is “If you tell the truth often enough people will eventually realize it is true and come to believe it”.
Yes, the appeals part only pertains to observations, not vam… And now the board of regents says the process has been fixed… Crazy.. I speak for all educators in wishing you two the best as you await the decision
There is a lot of discussion about litigation, but the reality is those of us in similar positions as Sheri don’t have Bruce Lederman (bless both of them), nor do we know how to find an attorney skilled in education law. The few lawyers that I have spoken to kind of laugh it off or others (who specialize in other areas) say that the good education lawyers either work for SED or for the school districts because that’s where the money is. This year, I was evaluated based on the scores of 5 students. Last year it was 3. There is nothing universal, fair, or equitable about this sham of a system.
The current NYS APPR law requires a teacher to have minimum of 16 students taking the test(s) in order for the scores to be used in your evaluation. Suggest you contact NYSUT legal.
Click to access appr-field-guidance.pdf
Oh yeah. I read through this with a fine-toothed everything. Our local and LRS from NYSUT is well aware, but claim “developing” doesn’t “mean anything”. It’s when we become “ineffective” that we need to start worrying. Hmm. Really?
(Reposted and Resynthesized)
It is only a guaranteed amount of time before legislation is passed to penalize parents big time if they opt their child out. This is sad on both ends, but it is the reality. The state can pass most any legisaiton it wants, and the only way to reverse bad laws will be the court system. Consequences will probably include but y no means are limited to tax refund penalties, child credit penalties, finical aid, Social Security disability aid, you name it; it’s all up for grabs and attachable to Opt-Out.
This is the Unite States where the customer is king and the citizen is nothing. Civil and rights and human dignity are up for sale in the clearance aisle of state legislators’ politics.
Will that be cash or credit? Next, please! May I help you?
It is sad and terrifying that it has come down to this, and from what I have read, the new APPR system mandates that districts fire teachers who, for two years in a row, have had scores of “ineffective”. It used to be an option to fire, but is now a mandate.
Teachers can only use a review or appeals process if they can make claims that districts acted fraudulently in their handling of the evaluation system. This is a very difficult thing to prove because it’s essentially like saying an administrator deliberately rated a teacher one way when in actuality, the adminsitator wanted to give a higher rating (I am referring to the observation process in this instance), but was directed or pressured to do otherwise because of other politics and agendas. What is said behind closed doors will never reach that teacher’s ears short of clandestinely placed, high tech surveillance devices, which are probably illegal to begin with and not feasible to get a hold of and implement.
As for test scores, there is no readily achiveable “fraud” to prove; the numbers are the numbers. To prove that the design of the tests and the utilization of scores is rigged or gravely flawed is something that is now being controverted in the Lederman case in NY.
I think it is very likely that scores will adversly impact teachers far more than observations because the incidence of administrator fraud is highly probable to be extremely low. Fraud only works against administrators and is really not in their interest. Any administrator worth their salt and who is witness to excellent teaching will want to retain that teacher, whether young, old, new, or veteran. Still, the new APPR system will destabilize school districts and create revolving doors. It will inevitably lower costs from a management and taxpayer perspective, but it will also severely jeopardize and compromise high quality teaching and learning from an instructional view.
As hard as this all is, one thing is guaranteed: The fight is anything but over, and most of it will be generated by parents, and not teachers and their unions. Although to see teachers act as private citizens is a game changer as well. Teachers should not have to rely on the parent movement. Teachers need very badly to reinvent their stale, lackluster, and corrupt, bloated unions that no longer truly serve them or the purposes of education either. Unions have become big machines whose DNA is now reprogrammed to ensure survival up at the top and the ability to collect full dues to finance and run the cogs of the machine that pays it top level executives’ unimaginable salaries and pensions IN ADDITION to their teachers pension. DOUBLE PENSIONS . . . . Now THERE’S a way to get union leaders to bow to the head honcho and vote her/his way every time. It’s easy to lose interest in one’s constituents when your head honcho union bosses incentivize you to gear your politics away from those you represent. It’s disgusting, and it’s the truth.
I feel like Donald Trump here ripping into the pure but bitter truth instead of presenting mannerly niceties with dressed up rhetoric. Not that the Donald is anything to brag about at all. Maybe I am waxing Bernie Sanders. Yeah. . . .That’s more like it.
It’s very sad, but it’s also bursting with hope, as new generations of Americans are starting to awaken from their shopping, refined sugar, and cable TV slumbers, all to find themselves as alert, conscious, civic participants.
With all its horrors, I think this is an exciting era.
For now, delays or no delays, this Board of Regents is a one way street, and it is more than clear the direction they are pointing to. I choose to have faith in Mrs. Tisch to do the right thing and really think about the fairness and utility in making 50% of an evaluation to be based on test scores. I am waiting and watching, hoping and anticipating, but the decision is up to her to think and act fairly.
When I say I have faith, I mean that I am expecting the other person to do the right thing simply because she has the ability and the vested authority to do so. There is no guarantee she will act accordingly, but I put the expectation upon her regardlessly in anticipation of something better, fair, and logical.
That is not at all to say that I don’t think other things should not be done either now or accordingly to how events play out. But I do believe in giving those in authority a chance to right the wrongs because that’s why they are there in the first place. I understand perfectly well their function in theory and in reality, and one of several things I can do right now is to hope the two shall merge.
But make no mistake that I am aware that one always needs more than hope.
If Dr. Ravitch can turn around, why not Dr. Tisch? I am waiting to see if this is a question of possibility vs. probability.
VAM has nothing to do with evaluation and everything to do with power to fire teachers.
Initially they figured surely principals would swing that scythe. When they refused, they accused nakedly all principals of being teacher lovers who weren’t doing their job so VAM will do it for them (ask Chancellor Farina how well the superintendent/network system functioned when you had the arm responsible for evaluating separate from the arm that was supporting – hint – networks didn’t matter because they had no power and the superintendent didn’t have the resources to help their principals align their goals with the super’s expectations)
Cuomo wants to fire bad teachers…they must be out there – after all students are not meeting the state’s expectations which go up and down more than a flag on a pole.
Here is a link to what I believe are the are regulations that the regents voted upon. The appeals process is on page 18. Not everyone is entitle to an appeal. Seems to me designed to preempt embarrassing cases like Sheri’s case, but not allow appeals in all cases.
Click to access 915p12hea1revised.pdf
Yeah as I recall reading you needed at least an effective on the evaluation portion and an ineffective
Surprising:
observations conducted by a principal or other trained administrator shall
be weighted at a minimum of 80%.
observations conducted by independent impartial observers shall be
weighted at a minimum of 10%.
if a district selects to use the optional third observation subcomponent,
then the weighting assigned to the optional observations conducted by
peers shall be established locally within the constraints outlined above.
So the independent evaluator is going to be the minority and the principal will count for the most? That is something since it will still make it so a principal can keep teachers from being fired simply by rating them effective since when the observation portion is rated effective, the worst the teacher can be rated is developing (thereby averting the worst of the consequences).
In a case like Sheri’s which they call an “anomaly” you can ONLY appeal if you are HIGHLY effective on the observation portion – it remains to be seen what the independent evaluator would need to score and what the principal to score to achieve “highly effective” on that component.
As for the “appeal” – it does appear Bruce intended to short circuit cases like yours going forward but there are 2 things you should note:
A) It ONLY applies to the 2014-15 school year – if I read your brief right, Sheri’s score is for 2013-14.
B) Quote “First, the Department has decided to reexamine the State growth model, which
will take additional time. In the interim, the Department has amended Subpart 30-2 and
30-3 to prescribe an appeals process whereby certain teachers or principals who were
13
rated Ineffective on their State-provided growth score may appeal to the Department
from their State-provided growth score based on certain anomalies described in the
regulation. The appeals process would apply to growth scores for the 2014-2015 school
year and thereafter until the growth model has been re-examined by the Department
and appropriate experts in the field. ”
So the appeals process is intended to be temporary until such time as they declare the formula “examined” with no criteria for what that means beyond consulting with experts (e.g. what benchmarks should this score be able to achieve before the appeals process is annulled)
Based on those 2 pieces, I think your case isn’t dead and there’s hope the whole thing can be declared nonsense.