Bruce Lederman is suing the state of New York on behalf of his wife Sheri Lederman, a fourth grade teacher in the public schools of Great Neck, Néw York. The Ledermans contend that the state teacher evaluation system is irrational, and Bruce collected affidavits from leading scholars to support his claim, as well as laudatory statements from students, parents, and Sheri’s principal and superintendent.
Alexandra Milletta, a teacher educator and high school classmate of Sheri’s, attended the trial and reported her impressions on her blog.
She wrote:
“What I witnessed was a masterful take down of the we-need-objectivity rhetoric that is plaguing education. So I should begin by saying that I am hopeful, because it seems someone with the power to make a difference gets it. Judge McDonough gets that it’s all about the bell curve, and the bell curve is biased and subjective….
“As you may notice, we’ve come a long way from getting a 91 out of 100 on a test and knowing that was an A-. Testing today is obtuse and confusing by design. In New York State, we boil it down to a ranking from one to four. That’s right, there’s even jargon for “ones and twos” that is particularly heinous when you learn that politicians have interests in making more than 50% of students fall in those “failing” categories. Today the state released the test score results for students in grades 3-8 and their so-called “proficiency” is reported as below 40% achieving the passing levels. By design the public is meant to read this as miserable failure.
“The political narrative of public education failure extends next to the teachers, who must demonstrate student learning based on these faulty tests, even if they don’t teach the subjects tested, and even if they teach students who face hurdles and hardships that have a tremendous impact on their ability to do well on the tests. In Sheri’s case, her rating plunged from 13 out of 20 points to 1 out of 20 points on student growth measures. Yet her students perform exceedingly well on the exams; once you are a “four” you can’t go up to a “four plus” because you’ve hit the ceiling. In fact, one wrong answer could unreasonably mark you as a “three” and you would never know. Similarly, the teacher receives a student growth score that is also based on a comparison to other teachers. When it emerged in the hearing today that the model, also known as VAM, or value-added, pre-determined that 7% of the teachers would be rated “ineffective” Judge McDonough caught on to the injustice that lies at the heart of the bell curve logic: where you rank in the ratings is SUBJECTIVE…..
The State’s representative, Colleen Galligan, tried to defend the indefensible:
“The lame explanation from Colleen Galligan was that the model may not be perfect but the state tries to compare each student to similar students. The goal, she offered, is to find outliers in the teaching pool who consistently have a pattern of ineffectiveness, to either give them additional training or fire them. At this point Judge McDonough offered her a chance to explain the dramatic drop in Sheri’s score. “On its face it must mean students bombed the test (speaking as one who has bombed tests)” and this produced laughter in the courtroom. For who hasn’t bombed at least one test in their life? Who has not experienced that dread and fear of being labeled a failure? Then Judge McDonough asked rhetorically, “Did they learn nothing?” The only answer she could come up with, was that in this case Dr. Lederman’s students, although admittedly performing well compared to other students, did worse than 98% of students across the state in growth. At this point it was pretty clear to everyone present that this made absolutely no sense whatsoever.”
Alexandra believes and hopes that this trial may be the beginning of the end for VAM and other misuses of test scores to rank and rate teachers.
Peter Greene did not attend the trial, but he cut to the chase: “God Bless Sheri Lederman!” I would add to that “God Bless Bruce Lederman” for fighting for his wife and her professional reputation. Together, the Ledermans are fighting for all teachers.
Peter read Alexandra Miletta’s post, cited above. He writes:
“The New York teacher is in court this week, standing up for herself and for every teacher who suffers under New York’s cockamamie evaluation system. If she wins, there will be shockwaves felt all across America where teachers are evaluated based on VAM-soaked idiocy….
“Talking about the curve is the best way to help civilians understand why these teacher eval systems are giant heaps of baloney. If you’re old enough, you remember curves because they suck– get yourself in a class with the smart kids who all score 100% on a test and suddenly missed-one-question 95% is a C. Of course, younger civilians may not have such memories of the curve because over the past few decades most teachers have come to understand that curving is not a Best Practice.
“Evaluating teachers on the curve means that even if the VAM-sauce score actually meant something, the teacher evaluation itself will not mean jack. In a system in which every single teacher is above the bar in excellence, those teachers who are the least above the bar will be labeled failures.”
Maybe one thoughtful judge will put the VAMMERS in their place: out of the classroom.

peter has once again hit the nail on the head! in college, I got the highest test score on an exam (in my section) – we took the test the day before the other section – the professor then added the results of the later section & I wound up with a C for the test – I’m sure that extra day (along with “blabbermouths”) made the difference – I wouldn’t have been so upset if I’d ultimately earned a B on the test, but a C? damn them bell curves!!!!
LikeLike
“damn them bell curves!!!!”
Full Speed Ahead!!!
LikeLike
Here’s more court coverage:
http://blog.timesunion.com/capitol/archives/239754/testing-opponent-says-system-flawed-judge-asks-if-arbitrary-capricious/
TIMES-UNION:
Sheri Lederman was back in state supreme court Wednesday with her husband and lawyer Bruce Lederman laying out his argument about why the state’s testing system used to help evaluate teachers is flawed and should be tossed out.
“The State Education Department does not get a pass on irrational and unreasonable actions simply because they…were getting a grant from the federal government,” Lederman told Acting Albany County Supreme Court Justice Roger McDonough.
“The magic of numbers,” Lederman added, “brings out the suspension of common sense.”
LikeLike
What is clear is that there has been a great deal of manipulation by the state to feed the failing schools narrative. There have been many concerns and complaints about the tests based on the CCSS including experts that have found the tests to be confusing and written on the frustration level of the grade level they purport to measure. In other words, they are designed to produce widespread failure. Parents should be outraged, not at the teachers, but at the cabal in Albany headed by Cuomo and Tisch, trying to destroy public education in New York. In addition, the state came up with a statistical model that ensures failure and potential dismissal for a great many competent, effective New York state teachers. Calling the formula flawed would be generous. I would call it gross negligence of the public trust. This type is of behavior is no different from rigging a horse race or an election. It is malfeasance of duty.
LikeLike
“That’s right, there’s even jargon for “ones and twos”
That was inevitable and predictable and ed reformers should have anticipated it. If they give people a 1-4 scale people will use it, particularly if they spend a year on a marketing campaign that claims this test is wholly reliable and definitive and people may safely use these numbers as proxy for everything under the sun.
Is it wise to go down this road with younger children? After all, they’re much younger than ACT/SAT takers. Might not want to label them in 3rd grade. Might be some unintended consequences there.
“Who cares! Full speed ahead!”
LikeLike
“Might not want to label them in 3rd grade. Might be some unintended consequences there.”
It’s not “might be” but “will be”.
7. And so what does this all mean? I’ll let Wilson have the second to last word: “So what does a test measure in our world? It measures what the person with the power to pay for the test says it measures. And the person who sets the test will name the test what the person who pays for the test wants the test to be named.”
In other words it attempts to measure “’something’ and we can specify some of the ‘errors’ in that ‘something’ but still don’t know [precisely] what the ‘something’ is.” The whole process harms many students as the social rewards for some are not available to others who “don’t make the grade (sic)” Should American public education have the function of sorting and separating students so that some may receive greater benefits than others, especially considering that the sorting and separating devices, educational standards and standardized testing, are so flawed not only in concept but in execution?
My answer is NO!!!!!
One final note with Wilson channeling Foucault and his concept of subjectivization:
“So the mark [grade/test score] becomes part of the story about yourself and with sufficient repetitions becomes true: true because those who know, those in authority, say it is true; true because the society in which you live legitimates this authority; true because your cultural habitus makes it difficult for you to perceive, conceive and integrate those aspects of your experience that contradict the story; true because in acting out your story, which now includes the mark and its meaning, the social truth that created it is confirmed; true because if your mark is high you are consistently rewarded, so that your voice becomes a voice of authority in the power-knowledge discourses that reproduce the structure that helped to produce you; true because if your mark is low your voice becomes muted and confirms your lower position in the social hierarchy; true finally because that success or failure confirms that mark that implicitly predicted the now self-evident consequences. And so the circle is complete.”
In other words students “internalize” what those “marks” (grades/test scores) mean, and since the vast majority of the students have not developed the mental skills to counteract what the “authorities” say, they accept as “natural and normal” that “story/description” of them. Although paradoxical in a sense, the “I’m an “A” student” is almost as harmful as “I’m an ‘F’ student” in hindering students becoming independent, critical and free thinkers. And having independent, critical and free thinkers is a threat to the current socio-economic structure of society.
LikeLike
I feel like parents were specifically told this WOULD NOT HAPPEN but every bit of coverage of the Common Core tests is all about passing and failing and 1-4.
They can’t keep telling people this is not all about the test score when every single thing they do and say is centered on the test score. They themselves compare these tests to the SAT/ACT. Those are high stakes tests administered to much older students and they are (mostly) voluntary. If they want an ACT for 8 year olds, the least they could do is admit it.
LikeLiked by 1 person
“the model may not be perfect”? No, it’s a joke. If one were to rate the various computer models in the world for modeling the weather, modeling structural materials, modeling traffic flow, or modeling educator effectiveness, which one would get an A, and which one would get an F?
People know the weather model is a rough guess, which is fine. But then why would VAM be taken as gospel? Bottom line, there is no value in VAM.
LikeLike
Absolutely, Peter Greene: “God bless” the Ledermans indeed.
VAM and its weirdo implementation across New York State has been an assault not only on the concept of free and public schooling but the idea of rational thought and the enlightenment. (Despite claims by its proponents that their mumbo jumbo is all so “scientific”.) It is truly “defending the indefensible”.
LikeLike
Assuming law is still based on logical arguments and rational thought rather than blatant political ideology and money, VAM has so many flaws and contradictions, this case should be an easy win for the Ledermans. The crumbling legal system on is trial. I just hope the judge, unlike the Atlanta trials, is competent enough to understand the arguments.
LikeLike
I don’t know if you-all have seen this- the inside picture of management approaches at Amazon- but you will recognize it. I read the investigator’s report on the Atlanta teacher scandal and the management approach there was identical, up to and including people crying at work constantly and huge turnover.
Anyway, take a look. I think we found the source of 90% of the ed reform policy that politicians and lobbying groups have fallen head over heels in love with. Amazon management techniques are like Patient Zero of the “market-based” approach that has infected our schools and captured all of DC and most statehouses.
You will immediately recognize this. It’s what they’re applying to all public schools, all over the country:
LikeLike
Amazon is famous for being a Darwinistic sweatshop. And they are expanding into Ohio.
LikeLike
Yeah, the article is horrifying to me. I will never buy anything from Amazon (as long as there are alternatives, which might not be long). If you do, you’re voting for this model of workplace organization. It’s an Ayn Rand utopia –weaklings kicked to the curb. Family, ethics, camaraderie….all thrown in the incinerator for the holy task of inflating Bezos’ ego, annihilating competitors that have more humane practices, and gratifying every last random twitch of customer desire instantaneously.
LikeLike
You do know that if the greed-is-good, for-profit corporate education RheeForm movement loses this case, they will appeal it all the way to the U.S. Supreme Court if they can and they have the deep pockets of the Waltons, the Koch brothers, the Bill Gates Cabal of thugs, Eli Broad and all the Hedge Fund billionaires to keep the lawyers coming from the most expense law firms.
And even if the RheeFormers lose at the U.S. Supreme Court, the odds favor that they will keep buying elections until they control the U.S. Congress, The White House and every state legislature and governor. I do not think they have any intention of stopping until they get what they want—-the United States and its lost democracy in their bank accounts.
I think that the RheeForm movement has already proven over the course of decades that these oligarchs will do anything to achieve their goals even if it leads to a bloody revolution in the U.S. and a global war that kills billions of people.
LikeLike
Lloyd,
I have a feeling Bruce Lederman will stand up for Sheri all the way to the US Supreme Court if need be
LikeLike
Love of your significant other is a powerful motivation, and most men want to defend and fight for their family.
LikeLike
Diane,
Any reason for the “awaiting moderation” for some of my comments?
TIA,
Duane
LikeLike
Obviously not this comment but the responses to FLERP!
LikeLike
Huummmm! My comments that are awaiting moderation have one thing in common, they have the word f l e r p ! in them.
LikeLike
Thanks, Diane, for the vote of confidence. I’ll be behind Sheri no matter what. If we need to go to the Supreme Court of the United States (or more likely the Court of Appeals of the State of New York), then Sheri and I will proudly do so together as a couple and as a team. Again, we thank all who have supported us and provided invaluable insight and support.
LikeLike
Sheri and Bruce,
Know how much hope, motivation, and strength you give to a few million people – educators and families alike – just by doing what you are doing and by being you!
Much support and positive energy,
Robert Rendo
LikeLike
Well, if they they win against VAM, it puts the DOE in a very bad position for appeal, because now you have a government agency fighting a ruling that states their policy is not just bad or wrong due to technicalities or circumstances, it is total crap. This will generate plenty of publicity and will look utterly ridiculous.
LikeLike
I agree that it will generate plenty of publicity, but the odds also favor that the major media will focus on demonizing those who are fighting against VAM in an attempt to turn public opinion against them—-a vigilante witch hunt reaction with death threats—-with an agenda to create a public backlash similar to what happened to that dentist who illegally shot Cecil the Lion.
In fact, newspapers like the New York Times will fake it that public opinion is against teachers and for VAM even if it isn’t true.
For instance, I just read yesterday that FOX news reversed public opinion about labor unions. It seems that a recent Gallop Pole revealed that 61% of the public thinks labor unions are necessary to protect workers rights compared to about 30% who didn’t, and when Fox news reported this, they reversed the figures and reported that 61% felt that the country didn’t need labor unions and only 30% felt we needed them to protect workers.
And FOX news wasn’t the only media outlet that did this. To fact check, readers would have to take the time—a few minutes probably—to Google Gallup and see for themsevles. When the average attention span on the internet is less than 30 seconds, I think that most of the easiest to fool readers are not going to fact check anything. Gallup also shows that about half of Americans believe anything they read or hear from the major media.
LikeLike
I love the judge’s question: “Did they learn nothing?” There is so much that my students learn that would never be measured on a Pearson ELA test. And even if I let the tests dictate what I taught: is there really a way to teach the skills the test allegedly tests for? I think this is a central question that few have really plumbed. I don’t think so. What we CAN teach is novels, poetry, grammar, history, geography, earth science (all of this is valuable, despite what the anti-knowledge ed school dogma claims). A standardized test could plausibly test kids’ knowledge in these areas (though inferring teacher quality from the results would still be dubious). But a standardized test, like the SBAC ELA, that aims to get at something “higher” than mere knowledge –now we’re really in thick fog. No one I know has cut through this fog.
LikeLike
“A standardized test could plausibly test kids’ knowledge in these areas. . . now we’re really in thick fog. No one I know has cut through this fog.
No, a standardized test couldn’t plausibly test kid’s knowledge due to construct validity issues.
And yes, someone has “cut through this fog” and it is Noel Wilson whose bright sunlight of rationo-logical thought has completely burned off the fog and destroyed the fog-making machine. Thanks for the lead in, Ponderosa! You know what’s coming next! I hope TE doesn’t get too upset again that I am posting it-ha ha!
“Educational Standards and the Problem of Error” found at: http://epaa.asu.edu/ojs/article/view/577/700
Brief outline of Wilson’s “Educational Standards and the Problem of Error” and some comments of mine.
1. A description of a quality can only be partially quantified. Quantity is almost always a very small aspect of quality. It is illogical to judge/assess a whole category only by a part of the whole. The assessment is, by definition, lacking in the sense that “assessments are always of multidimensional qualities. To quantify them as unidimensional quantities (numbers or grades) is to perpetuate a fundamental logical error” (per Wilson). The teaching and learning process falls in the logical realm of aesthetics/qualities of human interactions. In attempting to quantify educational standards and standardized testing the descriptive information about said interactions is inadequate, insufficient and inferior to the point of invalidity and unacceptability.
2. A major epistemological mistake is that we attach, with great importance, the “score” of the student, not only onto the student but also, by extension, the teacher, school and district. Any description of a testing event is only a description of an interaction, that of the student and the testing device at a given time and place. The only correct logical thing that we can attempt to do is to describe that interaction (how accurately or not is a whole other story). That description cannot, by logical thought, be “assigned/attached” to the student as it cannot be a description of the student but the interaction. And this error is probably one of the most egregious “errors” that occur with standardized testing (and even the “grading” of students by a teacher)
.
3. Wilson identifies four “frames of reference” each with distinct assumptions (epistemological basis) about the assessment process from which the “assessor” views the interactions of the teaching and learning process: the Judge (think college professor who “knows” the students capabilities and grades them accordingly), the General Frame-think standardized testing that claims to have a “scientific” basis, the Specific Frame-think of learning by objective like computer based learning, getting a correct answer before moving on to the next screen, and the Responsive Frame-think of an apprenticeship in a trade or a medical residency program where the learner interacts with the “teacher” with constant feedback. Each category has its own sources of error and more error in the process is caused when the assessor confuses and conflates the categories.
4. Wilson elucidates the notion of “error”: “Error is predicated on a notion of perfection; to allocate error is to imply what is without error; to know error it is necessary to determine what is true. And what is true is determined by what we define as true, theoretically by the assumptions of our epistemology, practically by the events and non-events, the discourses and silences, the world of surfaces and their interactions and interpretations; in short, the practices that permeate the field. . . Error is the uncertainty dimension of the statement; error is the band within which chaos reigns, in which anything can happen. Error comprises all of those eventful circumstances which make the assessment statement less than perfectly precise, the measure less than perfectly accurate, the rank order less than perfectly stable, the standard and its measurement less than absolute, and the communication of its truth less than impeccable.”
In other word all the logical errors involved in the process render any conclusions invalid.
5. The test makers/psychometricians, through all sorts of mathematical machinations attempt to “prove” that these tests (based on standards) are valid-errorless or supposedly at least with minimal error [they aren’t]. Wilson turns the concept of validity on its head and focuses on just how invalid the machinations and the test and results are. He is an advocate for the test taker not the test maker. In doing so he identifies thirteen sources of “error”, any one of which renders the test making/giving/disseminating of results invalid. And a basic logical premise is that once something is shown to be invalid it is just that, invalid, and no amount of “fudging” by the psychometricians/test makers can alleviate that invalidity.
6. Having shown the invalidity, and therefore the unreliability, of the whole process Wilson concludes, rightly so, that any result/information gleaned from the process is “vain and illusory”. In other words start with an invalidity, end with an invalidity (except by sheer chance every once in a while, like a blind and anosmic squirrel who finds the occasional acorn, a result may be “true”) or to put in more mundane terms crap in-crap out.
7. And so what does this all mean? I’ll let Wilson have the second to last word: “So what does a test measure in our world? It measures what the person with the power to pay for the test says it measures. And the person who sets the test will name the test what the person who pays for the test wants the test to be named.”
In other words it attempts to measure “’something’ and we can specify some of the ‘errors’ in that ‘something’ but still don’t know [precisely] what the ‘something’ is.” The whole process harms many students as the social rewards for some are not available to others who “don’t make the grade (sic)” Should American public education have the function of sorting and separating students so that some may receive greater benefits than others, especially considering that the sorting and separating devices, educational standards and standardized testing, are so flawed not only in concept but in execution?
My answer is NO!!!!!
One final note with Wilson channeling Foucault and his concept of subjectivization:
“So the mark [grade/test score] becomes part of the story about yourself and with sufficient repetitions becomes true: true because those who know, those in authority, say it is true; true because the society in which you live legitimates this authority; true because your cultural habitus makes it difficult for you to perceive, conceive and integrate those aspects of your experience that contradict the story; true because in acting out your story, which now includes the mark and its meaning, the social truth that created it is confirmed; true because if your mark is high you are consistently rewarded, so that your voice becomes a voice of authority in the power-knowledge discourses that reproduce the structure that helped to produce you; true because if your mark is low your voice becomes muted and confirms your lower position in the social hierarchy; true finally because that success or failure confirms that mark that implicitly predicted the now self-evident consequences. And so the circle is complete.”
In other words students “internalize” what those “marks” (grades/test scores) mean, and since the vast majority of the students have not developed the mental skills to counteract what the “authorities” say, they accept as “natural and normal” that “story/description” of them. Although paradoxical in a sense, the “I’m an “A” student” is almost as harmful as “I’m an ‘F’ student” in hindering students becoming independent, critical and free thinkers. And having independent, critical and free thinkers is a threat to the current socio-economic structure of society.
LikeLike
Typo between 4. and 5.
In other word(s)
LikeLike
Thanks, corrected on this end!
LikeLike
On another thread, I shared some thoughts on this topic. It was a bit rambling at times… but here it is:
I just read about the concept of “growth” in the Leaderman case, and some thoughts came to mind…
First, there’s the phenomenon of the “ceiling effect” when judging someone based on a growth model of data. This is a handicap where, in raising your “growth” data, you have nowhere or almost nowhere higher to go, or there a circumstances present that make it very difficult to go higher. This might be the case with Leaderman’s students. They scored massively high in two consecutive years, with a miniscule dip from one year to the next.
In Los Angeles, I’ve looked up the stats on two exemplary public schools in upscale neighborhoods: Wonderland Avenue up in Laurel Canyon, and Warner Avenue in Westwood.(the school where privatization puppet Ben Austin sends his kids, by the way.)
In their annual API score(maximum score 1000), these two schools consistently fluctuate between in a range between 980-995, but never 1000. They’re at the “ceiling.” Within that range, their scores go up or down 5-to-15 or so API points from year to year. It’s impossible for those schools to show much, if any “growth”. They’re at the ceiling, and that makes moving up the score almost impossible—you have nowhere to go but down. When their score dips slightly—say 990 down to 980—that would lead to them having a low “growth” score.
On the other hand, a struggling inner-city school might start with an API score of 600 one year, that goes up to next year to 620, then 630 the year after. (When a school starts out this low, these gains are fairly easy to attain, much easier than at Wonderland or Warner.) In the “growth model” world, this school would have a high growth score, while Warner and Wonderland would be “outliers”, with a poor “growth” score.
Thus, the growth model is nonsense.
Here’s how “ceiling effect” works in sports. I’m not sure, but Ms. Leaderman—in regards to her students’ scores year-to-year—may be sort of in the position of former Lakers’ coach Phil Jackson was in the early 2000’s, when his teams won 70 or more games in multiple seasons.
Keep in mind that team winning over 70 games is a season is rare in NBA history.
The argument goes like this: If the team that Jackson coaches wins, for example, 74 games one year, then only 71 games the following year—again, both phenomenal achievements—he would then get an “ineffective’ rating on his team’s “growth.” Since the system is a bell curve, one could mandate that 7% of all coaches MUST be given an “ineffective” rating… no matter what. He might be in that group.
Conversely, a coach who went from, say, 11 wins one year, and then “grew” to 14 wins the next year would be awarded a “highly effective” growth score… even though he would be benefiting from the opposite phenomenon— the “floor effect”… the opposite of the”ceiling effect” hampering Jackson. Thanks to the “floor effect,” this coach had nowhere to go but up, and that fact made it a lot easier to do so than in Jackson’s case.
The variable in both of those cases is, of course, the players… the analogy in education being the students. The quality of the students a teacher receives–and more importantly, the quality of the parents… are they involved, college-educated, etc.? … is a big piece of this puzzle.
Let’s look at other variables in other jobs.
There are some jobs where the person performing it has total control over the outcomes.
For example, a mailman is given a square half-mile or so that he has deliver mail to every day. While there may be variables inhibiting his job performance—weather, dogs blocking access to the mailboxes, whatever—for the most part, that mail carrier is in almost 100% control of the conditions that would lead to him either performing his job well or poorly. If he fails to deliver mail to the entirety of his territory every day, there’s no one to blame but him.
Another example could be a gourmet “made-by-hand” cookie maker, who has the job of making sure that exactly 20 jumbo chocolate chips are in each of the jumbo chocolate chip cookies he or she makes by hand (this is making me hungry 😉 ). In this case, the worker is again in 100% control over producing the quality demanded of him—20 chips-per-cookie.
If the mail carrier or cookie maker doesn’t deliver the results demanded of him or her, he or she has no one to blame but himself or herself.
I could present endless such examples.
On the other hand, a teacher’s job is not like that. It’s more akin—but not exactly akin, mind you—to that of a dentist’s, or say, a personal fitness trainer’s. In those cases, no matter how talented or inspiring you are in your job, your student / dental patient / fitness client may lack the same dedication or innate ability to deliver optimum results. Not all students / dental patients / fitness clients are equal… a factor beyond the control of the teacher/dentist/fitness trainer.
Now what do I mean by innate ability? Let’s examine the dentist’s dilemma. Based on the chemical makeup of a person’s mouth and saliva, some people are more prone to get cavities—given the same foods consumed by the patient. Those patients are “special” ;-). They will have to be a lot more aggressive in their home care, diet, etc. to avoid cavities. In addition, their outcomes also dependent on how they are supported by the family members they live with.
Other patients, for the same reason, will be innately and significantly less prone to cavities, and thus, will have to work less hard, and not have to be as prudent in their diet… to achieve the same result.
That’s beyond the dentist’s control.
The other thing beyond the dentist’s control is the same thing that’s beyond the personal fitness trainer’s control. At each visit, the dentists can give the patient the best pep talk ever, demonstrate proper brushing and flossing technique, and provide him with free brushes, toothpaste, floss, ACT anti-plaque mouthwash, etc. as the patient walks out the door.
However, after that, it’s up to the patient to deliver and complete the “homework” which the dentist assigns to that patient. If he or she never or rarely brushes/flosses his teeth, never washes with ACT, or eats too many jumbo chocolate chip cookies (I’m getting hungry again 😉 ), is it the dentist’s fault when the patient comes back with cavities… resulting in fillings, root canals, and tooth extractions?
Would we have ratings for dentists, or pay those dentists based on such “data”? Of course not, but that’s what Campbell Brown and so-called “corporate refomers” want for public school teachers (but strangely, not for her own kids, who attend Heschel, a rich kids’ private school where none of the teachers are evaluated based on students’ test scores.)
“Your patients had more cavities, so we’re paying you less, and if you don’t improve, you’re fired… And you over there, your patients had less cavities, so we’re paying you more, and giving you a promotion.”
Does that make sense? I didn’t think so.
As for the personal fitness trainer, I’ve spoken to them and the qualities of their clients also vary wildly, and not just in their innate ability, but also in their mental determination and performance during 1-on-1 training. Some clients approach these sessions with a ferocity that matches the trainer’s. These clients always give more, or do more than is asked of them. This also is hopefully reinforced in the home environment—by their spouses, significant others, or family members… another factor beyond the control of the fitness trainer.
Could you imagine if the fitness club manager said that they were going to measure the beginning state of the client’s fitness—weight, muscle mass, heart rate, etc.—and then 9-10 months later, re-measure all of that, and the judge the trainer’s effectiveness on the “growth” or improvement in the client’s fitness? Some clients will improve. Some will not. But in the case of the latter, how much is it the fault of the fitness trainer? Should a trainer be punished or rewarded based on such a “growth” system?
Indeed, some clients will not progress or show “growth,” no matter how talented or inspiring the trainer is in the performance of his job. In the context of a training session, these clients don’t even want to break a sweat, and barely do a thing, instead content to complain to the trainer about their spouses or their jobs or their kids, or whatever. With the latter, they may as well stay at home, for all the good the training is doing…. another factor beyond the control of the fitness trainer.
Furthermore, it may be the case that no one in the client’s life outside of the gym—spouses, significant others, family members, etc.—is encouraging them, or reinforcing their efforts to get in shape, and stay in shape. Their family members a’re not doing so, and may NEVER have done so in their lives… and so they might not even want that client to excel in the first place. Misery loves company.
This leads to the next part… the homework.
A fitness trainer might train someone Monday-Wednesday-Friday, while giving his client the “homework” he should do on Tuesday-Thursday-Saturday-Sunday—i.e. 30-60 minutes cardio, 30-60 minutes weights… and most important, maintain a good diet… the real killer for some people’s efforts to get in shape.
If the client does little or none of the “homework,” then also gorges himself on Hostess Cupcakes, slabs of fatty ribs, baskets of seasoned curly fries, buckets of pasta with rich sauces, gallons of ice cream, etc. (Are you hungry yet? 😉 ), should the fitness trainer be held responsible for the poor results of the client?
Of course not.
To carry this analogy further, if fitness trainers were judged this way, this would lead to them avoiding certain potential clients, or even whole gyms. There’s a Gold’s Gym near my house with a predominantly gay clientele. For the most part—excuse the stereotype—the gay men are much more finicky about keeping their bodies fit. They work extremely hard in the gym and they watch what they eat. (Oh how I envy this attribute;-).) In this hypothetical scenario, all the trainers whose pay is merit-based are going to flock to work here. Many middle aged clients start out in poor shape, so for the trainer, you have the “floor effect” at work
Meanwhile, regarding another other gym with—excuse the stereotype again—hopelessly apathetic, overweight straight guys who can’t or won’t do the work, both at the gym and at home… no trainer being paid or fired on merit will want to go anywhere near that gym and train those folks. Evaluating the trainers who have these men as clients based on “merit”??? Would that be fair?
To carry that analogy further, some individuals—straight or gay— or groups are genetically pre-disposed to be overweight (Samoans for example) and have greater difficulty getting into shape, or staying in shape. No trainer who’s being paid based on merit will ever want to have those “special” folks as their clients.
That’s all I’ve got to say for now.
LikeLike
I believe a Florida teacher made the same observation with his highest achieving students. They literally had nowhere to go in the statistical construct. This is why it cannot be used to judge the performance of a teacher. The formula is both arbitrary and capricious. Do you think careers should be destroyed over such nonsense?
LikeLike
“. . . that they were going to measure the beginning state of the client’s fitness—weight, muscle mass, heart rate, etc.—and then 9-10 months later, re-measure all of that, and the judge the trainer’s effectiveness on the “growth” or improvement in the client’s fitness?”
Now, those “beginning states” of fitness can actually be measured. Weight, BMI, heart rate, etc. . . all have agreed upon either metrological standards as for weight or documentary standards such as BMI and heart rate from which to validly gauge the changes over time.
There are no such agreed upon standards, whether metrological or documentary which are agreed upon by educators, therefore there can be no “measure” of change, growth and/or decay. So that these VAM/SGP schemes have no basis in logical valid rationality at the conceptual basis, in other words the epistemological and ontological underpinnings are invalid for those educational standards and the supposed (actually false) measuring devices-standardized tests.
LikeLike
In general, when I’m working out, I want to lose fat, and gain muscle mass… which hopefully will slow down my heart and breathing rate…
That said, it’s equally absurd to do a Before & After measurement of those things for the purposes of assessing a fitness trainer’s effectiveness or lack thereof, as it is to do the same Before & After measurement of student’s test scores, and then used that to rate a teacher.
I remember when this VAM nonsense broke out in the L.A. Times in 2010.
Across the country in NYC, an anti-VAM teacher/blogger, South Bronx Teacher called up Richard Buddin, the expert non-teacher economist and statistician that the pro-VAM L.A. Times’ writers had hired to VAM-ify and assess thousands of LAUSD teachers based on their students’ test scores. These results were later published in a database, along with those teachers names. Teachers were assigned a label on a 5-degree scale: HIGHLY EFFECTIVE, MODERATELY EFFECTIVE, NEUTRAL, MODERATELY INEFFECTIVE, and HIGHLY INEFFECTIVE.
This led to one LAUSD teacher, Rigoberto Ruelas, reacting to this public shaming by committing suicide,
When questioned by South Bronx Teacher, Buddin conceded that this measurement was not precise enough to judge teachers and assign such labels, in part because there were other factors that VAM doesn’t measure.
Jason Felch, one of the three Times writers then later responded angrily to that blogger, South Bronx Teacher, and angrily posted that South Bronx Teacher had ambushed Buddin, and denied Buddin said that.
For anyone interested, here’s a flashback to that incident:
http://www.southbronxschool.com/2010_08_01_archive.html
————-
SOUTH BRONX TEACHER:
One lead we did follow was one of the writers of the story, Jason Felch. Jason was kind enough to share the wonderment of himself with us here at SBSB. However Jason’s journalistic integrity has come into question. Or has it? It is, of course, all in the context of what integrity is.
But enough of Jason Felch. He is just the messenger of evil. Jason needed to get someone, somewhere to crunch those numbers. Someone nefarious and someone who is getting paid big bucks.
That someone is Richard Buddin of the Rand Corporation and UCLA. One would assume that Mr Buddin would have had some type of background in education. After carefully scrutinizing his resume no one on the crack SBSB team was able to locate any such experience. Seems Mr Buddin is nothing more than an average, everyday economist.
First contact was made yesterday with Mr Buddin. The identity of the blog was never hidden, in fact it was out there. He was amiable enough to take time from his busy brown bagged lunch to talk with us here at SBSB.
We asked Mr Buddin if he had any background in elementary or secondary education. Mr Buddin answered with a resounding “no.” But, and this was quite reassuring, Mr Buddin did share that he does teach at UCLA. Goosebumps is all that can be said.
Mr Buddin was asked if he had ever been in an urban area. He reply was negative, he had not. Was he ever in Compton, East LA, the Barrio, Watts. No, no, no, no went the answers.
He was asked, “Did you take into account the backgrounds of the students in using value added assessment?” He said he had not.
When asked “why and how could he do this and at the same time ruin several teachers careers?” he replied, and I quote, “The study was supposed to be broad statistical analysis and at no time did he suspect individual teachers were to be used as an example.”
This led to him being reminded that even a 7th grader learns in science that when doing an experiment to have all the facts available before such experiment. At this point Mr Ruddin accused yours truly of having some type of agenda and then hung up like an 8th grade girl finding out her best friend is also in love with David Cassidy.
Since we here at SBSB believe in true journalistic integrity, as compared to the Los Angeles Times, we decided to contact Jason Felch for a response.
Jason, looking down his nose at the blogging world, initially claimed he was to busy. Mr Buddin’s words were texted to Jason. When Jason was contacted later on, he said again he had to make deadline. But he had time to say that he had spoken with Mr Buddin.
Jason Felch said I fibbed. He had spoken with Mr Buddin and Mr Buddin told him that those words were never uttered. I explained that I quoted him verbatim. Jason then, in his best patronizing, condescending manner explained to me how to be a writer, a journalist.
I explained, then asked Jason if he meant if this were akin to taking things out of context. Cat got his tongue.
SBSB stands behind what Mr Buddin said. He said it, he meant it, and he got caught up in it.
Jason, you and Mr. Buddin deserve one another.
————————————–
Anyway in 2014, four years later, there’s a coda to this story.
That VAM cheerleader, Jason Felch, was later fired by the L.A. Times for having sex with one of his sources, with the Times announcing that Felch had skewed the story in a way that was more to that source’s liking, all in exchange for sex.
Needless to say, South Bronx Teacher called this stunning denouement “karma” for Felch, who was married and a father at the time:
http://www.southbronxschool.com/2014/03/karma-smacks-los-angeles-times-reporter.html
————————————–
SOUTH BRONX TEACHER:
“Karma usually has a way of coming back and kicking you in the butt. This week Los Angeles Times reporter Jason Felch felt Karma’s full wrath.
“For those that do not recall, Jason spearheaded a Los Angeles Times report in August of 2010 using data of that funky science of value added measurements to determine who is and isn’t an effective teacher. Jason took great pride in doing his part reporting what he saw out of context, but to destroy careers as well … ”
and on it goes…
LikeLike
It sounds like they have a good and sympathetic judge, but don’t forget that this is a very high standard that Lederman has to meet.
LikeLike
One real life example of the absurdity of testing. Watkins Glen School District (upstate NY) used the STAR Literacy to measure gains in reading ability, factors into APPR /teacher score accountability results. My son maxed out this test in 5th grade, but was required to take it 21 more times. He is a very smart kid – so towards the end he used the test for his own means of grading the school. He would log in, choose the teacher he felt was among the worst, and proceeds to take the test with a resulting grade of first grade reading ability. There you, you have been VAMMED by your own assessment.
LikeLike
Ah, glimpses of the hidden unspoken curriculum that permeates the teaching and learning process. What we as teachers would like the students to learn isn’t always what the students learn for themselves.
Oh, we have to teach them “grit” and “stick-to-itness” and the student learns about tyranny and lynchings that continue until they supposedly learn that “grit” and “stick-to-itness”
LikeLike
Maybe your son could testify at a future trial about all this VAMiness?
LikeLike
“The more any quantitative social indicator (or even some qualitative indicator) is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.” [Campbell’s Law]
Campbell’s Law means that human beings, as opposed to things or unconscious beings, will do whatever they must to circumvent the logic of a Bell Curve, because the logic of a Bell Curve is a zero-sum logic, which means only a fixed percentage of people can succeed, period. Individual human beings, of course, will not accept this logic, and in pre-emptive strikes against zero-sum situations, individual human beings will corrupt the reality the Bell Curve supposedly describes in order to reduce the likelihood that a judgment of failure will fall on them. (a big DUH!)
So, even if SED were correct in presuming that 7% of teachers must be incompetent, [which it is not], its VAM model cannot identify exactly who these 7% are, because, following Campbell’s Law, teachers and principals and parents will try to corrupt the random distribution of students, and teachers and principals will insist on teaching to the test, to reduce the likelihood that they will fall within this 7%. These rational responses from teachers, principals and parents, therefore, will negate the reality the Bell Curve claims to describe (namely who the hell is a good teacher?). Hence VAM can never work because it is contradicted by Campbell’s Law. (Teachers and principals will invent all sorts of maneuvers to avoid the death-grip of the dreaded 7%, just as SED now uses all sorts of intellectual legerdemain to convince us that its use of VAM is absolutely free of corruption!)
The corruption of the use of Bell Curve analysis in human affairs is seen most famously in the corruption of the distribution of SAT scores due to the influence of tutors to train kids to take the SAT, (hurrah for Stanley Kaplan, the first to recognize this truth and to make a killing using it as a commercial idea–read The Big Test to learn about this fascinating tale and the embarrassment the College Board has endured because of it), or outright cheating, or the use of income distribution to construct questions that will produce a Bell Curve spread of answers.
More and more colleges are coming to accept the reality that Campbell trumps Bell, which means finding “worthy” college candidates demands the use of better “metrics” than SAT scores.
Once human beings know that authorities have decreed that some of them must fail at some endeavor, many will stop at nothing to avoid such a fate, especially if failure means the loss of a job, or the loss of an education, or the loss of income. This is Human Being 101.
The application of natural science insights to human affairs is very old, and powerful people who cannot convince ordinary people of the virtue of some course of action, especially if that course involves things like dying in war or denying children a decent education, will use so-called “natural science” realities to shield their impotence, since “natural science” means something human beings cannot change. But most ideas from the physical and mathematical sciences do not work all that well in the realm of social science because as soon as insights about how people behave become known, people begin to change social reality to prevent the negative effects of such an insight from affecting them. It’s why mathematical models used on Wall St go only so far in helping investors make money. (Outright fraud is much better at generating great returns!)
So we are reminded of the law of “unintended consequences of purposive social action.” Game theory is one science that studies this special reality of human action (as opposed to the action of meteors and squid).
It might be interesting for the Lederman’s to introduce appropriate insights from game theory to educate the Judge still further about the stupidity of VAM.
Campbell trumps Bell. Knowing this reality to be so, watch SED and the Rheeformers squirm, lie, and confuse to get their way. (Not to mention bribery.)
Go Lederman!!
LikeLike
Excellent commentary, Steve,
My only little beef is with one little phrase, that sums up what you are talking about:
“But most ideas from the physical and mathematical sciences do not work all that well in the realm of social science. . . ”
Exactly, but I don’t consider economics, anthropology, sociology, etc. . ., to be “sciences” but “studies”. In other words, social studies and not social sciences. I reserve the science moniker for those physical and mathematical realms of knowledge that are backed by “hard” scientific principles.
LikeLike
I also tend to disagree that the the main focus in this case should be on the “bell curve” aspect of the evaluation law. The bell curve may not be a best practice, but it’s been used for a long time and remains widely used. I would be very surprised if the court were to find that the law is arbitrary and capricious based solely on the fact that it uses bell curving.
LikeLike
I would like to know what constitutes an “outlier.” If turns out the outliers are those teachers whose students do not fit the typical growth patterns such as teachers of compensatory services in math or reading, ESL or special education teachers there should be further action taken such as a class action suit as all these teachers in the state cannot be incompetent!
LikeLike
FLERP!,
It seems to me that the case will rest on whether the evaluation system will be ruled “arbitrary and capricious” which I believe is the wording of the law that the Ledermans are using to discredit the state and it’s scheme. Can you please give us a bit of a primer on what those words mean and how does one know if the “mythical” threshold has been reached for those terms?
Gracias,
Duane
LikeLike
“Arbitrary and capricious” is the standard of review in a proceeding in New York state court to challenge a discretionary decision of an administrative agency. It’s called an Article 78 proceeding. The general idea is that administrative agencies — like the NY State Education Department, or the NYPD, etc. — should be given a certain amount of latitude to run their internal affairs, both (1) those agencies are better-positioned to make these decisions than courts are, and (2) both the administrative state and the court system would be unable to function properly if every action by an agency were subject to direct review by the courts. So if you’re an employee of an administrative agency, and the agency takes some action against you, you can’t just go challenge that in court. You first have to ask the agency to review its own decision, pursuant to whatever process the agency’s regulations require. If the agency refuses to change its decision, then you can file a “petition” in a NY court under Article 78. The court will review the agency’s decision, but it will give the agency a lot of deference. That deference is the “arbitrary and capricious” standard.
Courts don’t do a great job of explaining what it means for something to be “arbitrary and capricious,” but essentially it means that the agency’s decision has to have no basis in reason or fact. That’s why I used the term “irrational” above. And “irrational” truly means irrational in this context. It doesn’t mean that a decision is bad or unfair or unwise. It means that it literally has no rational relationship to the goal that the agency was trying to achieve with its decision.
I look at it like this: The petitioner stands up and tells the judge that the agency’s decision is arbitrary and capricious for reasons X, Y, and Z. The judge then says to the agency and says, “Look, you know the drill. Can you explain to me how your decision could make sense in any universe? Is there *any* reason why a sane person would make that decision?”
Having put it that way, I would add that there is a lot of grey area in there, and a judge who thinks that a decision or policy is terrible or idiotic definitely has the leeway to grant a petition, if he or she is inclined that way.
LikeLike
Thanks, FLERP!
Contextualized and crystal clear.
LikeLike
Diane,
Any reason my comment to FLERP! is “awaiting moderation”?
LikeLike
Duane, I went out to lunch with friends. Patience is a virtue.
LikeLike
Thanks, I didn’t think that you moderated all comments. As stated above the commonality that I found was f l e r p !’s screen name in the post.
Hope you enjoyed your lunch!
LikeLike
Steve: Thank you for thoughtfully articulating something that has been occurring to me as this case has wondered through the system. We are not submitting further papers, at this point. However, I will keep this issue of game theory in mind if we end up on appeal.
LikeLike
Thanks for all you are doing.
Please also keep in mind just how invalid educational standards and standardized testing is as explained by Noel Wilson. If you haven’t read his work please do so as it will help to understand just how completely invalid these processes are. See: “Educational Standards and the Problem of Error” found at: http://epaa.asu.edu/ojs/article/view/577/700
LikeLike
Also for a shorter version on the validity issues see Wilson’s essay review of the testing bible: A Little Less than Valid: An Essay Review
Click to access v10n5.pdf
LikeLike
FLERP!. FYI — the Bell curve was not the main focus of the case, but it was what seemed to catch the Judge’s attention. When you litigate, you often have multiple theories and you never know what will influence the Judge. Also, even though the Judge seemed most interested in the Bell Curve issue, when he goes to read all the papers and write a decision, he may focus on something else. Below is a link to the main papers. If you want to understand what issues we were arguing, please read the document entitled REPLY MEMORANDUM and you will see that Bell Curve is just an aspect of the entire presentation.
https://www.copy.com/s/t%3AhfUPvSmzK3MdHG3G%3Bp%3A%252FLederman%2520v.%2520King
LikeLike
Thanks, Bruce. Best of luck.
LikeLike
In 1975 as a college frosh, every intro class (phycology, sociology, anthropology et al) had 500 students in huge lecture halls. Every course had bell curve tests and almost everyone esrned Cs. Not a good way to set your GPA baseline……seemed unfair. Never heard about use much since. It took years of smaller classes with authentic interactions with professors to raise our grades. Teachers with VAM do not have the ability to change their rating.
LikeLike
“The bell curve may not be a best practice, but it’s been used for a long time and remains widely used.”
So in law, this is reason enough to continue to use a discredited and flawed system?
LikeLike
FLERP, I did not get the impression that the Lederman’s testimony added up to “OMG bell curve = bad” but rather that this was an inappropriate use for that kind of measure. When I took Regents tests (decades ago) it was theoretically possible for everyone in my year to pass , or even excel, on one test. That set might be a more appropriate measure of how well students had mastered a particular range of material.
LikeLike
The question may turn on whether the use of that kind of measure is “inappropriate” because there are better ways to evaluate teachers than by using that kind of measure, or because there is no possible rational reason why the state would use that kind of measure to evaluate teachers. This is the high bar I referred to above.
LikeLike
“Measuring outcomes through standardized testing, and referring to those results as the evidence of learning and the bottom line is, in my opinion, misguided and unfortunately continues to be advocated under a new name, and supported by the current (Obama / Duncan) administration.
“Physical education, world languages, libraries, and the the arts are not ‘frills.’ They are and essential piece of a well-rounded education.
“I shudder to think of who would be attracted to teach in our public schools without unions.”
— Dr. David McGill
Director of the University of
Chicago Lab School since 2002
(where Barak Obama, Arne Duncan,
and Mayor Rahm Emanuel sent/send their
children)
Quoted by Chicago public school parent /activist
Matt Farmer in a speech he gave back in 2012,
captured in the following video (the McGill
quotes start at around 4:35)
==================
Here’s some neoliberalism at work in the Chicago schools, and one public parent giving a speech calling out those behind it.
The Matt Farmer YouTube video is a great video that shows the double standard they have — what kind of education that the wealthy elite think their own kids deserve—everything and then some,— compared what they think the children of the middle and working classes deserve—just enough education to function as a working drone.
First of all, Chicago doesn’t have an
elected school board. What they DO
have is akin to a rubber-stamp
phony parliament or legislature in a
Third World dictatorship—one
appointed by the Mayor without
any input from citizens.
It’s made up of business leaders with ZERO
background in education as teachers
or administrators, or anything. They
actually funnel ed money out of
education into TIF funds, which are
then used to subsidize the building of
high-rise hotels constructed by some
companies owned by someone on
the board—i.e. former Chicago
Schools Board Member and Hyatt
Hotels executive Penny Pritzker.
It’s so corrupt it staggers the imagination.
Check out parent activist Matt Farmer
calling out then Board Member Penny
Pritzker for her hypocrisy in gutting the arts,
phys. ed., libraries, etc. from the
traditional public schools, while
simultaneously raising millions for a
new library and other facilities
at the Chicago Lab School where
her own children attend.
He begins by quoting from an
interview where Pritzker states that
that the traditional public schools—
where her kids do not attend, but
the children of middle and working
class kids do—are only responsible
for providing the bare minimum
required to perform at low level
jobs in the workforce… and no critical
thinking education, God forbid!…
and that’s all that Pritzker believes
that the children of the middle and
working classes “are entitled to.”
Matt then brings the facts, and
brings the fire. Since Matt is
a lawyer (and journalist), he
“cross-examines” Pritzker in
abstentia. In the process, he
delivers one of the greatest
speeches against so-called
“corporate reform” and
privatization ever given.
It’s a classic:
LikeLike
Where is the judge’s decision on the leader an case.. It was supposed to be 2 or 3 months for that decision and it has been more that 4. Keep checking Internet and newspapers and nothing. Does anybody know?
LikeLike
Sorry I didn’t check my comment my misspell changed Ledermen to leader. Has any heard the judge’s decision.
LikeLike