Paul Thomas follows Anthony Cody’s previously cited post by describing the unrelenting attack on teachers, which has intensified with the use of statistically inappropriate measures.
He writes:
“As Cody notes above, however, simultaneously political leaders, the media, and the public claim that teachers are the most valuable part of any student’s learning (a factually untrue claim), but that high-poverty and minority students can be taught by those without any degree or experience in education (Teach for America) and that career teachers no longer deserve their profession—no tenure, no professional wages, no autonomy, no voice in what or how they teach.
And while the media and political leaders maintain these contradictory narratives and support these contradictory policies, value-added methods (VAM) of evaluating and compensating U.S. public teachers are being adopted, again simultaneously, as the research base repeatedly reveals that VAM is yet another flawed use of high-stake accountability and testing.”
Thomas cites review after review to demonstrate that VAM is inaccurate and deeply flawed. Yet the evidence is ignored and VAM is being used as a political weapon by the odd bedfellows of the Obama administration and rightwing governors as well as some Democratic governors, like Andrew Cuomo of New York and Dannell Malloy of Connecticut, to attack teachers. President Obama made a point of praising the Chetty study in his 2012 State of the Union address, not waiting for the many reviews that showed the error of measuring teacher quality by test scores.
Thomas writes:
“The rhetoric about valuing teachers rings hollow more and more as teaching continues to be dismantled and teachers continue to be devalued by misguided commitments to VAM and other efforts to reduce teaching to a service industry.
“VAM as reform policy, like NCLB, is sham-science being used to serve a corporate need for cheap and interchangeable labor. VAM, ironically, proves that evidence does not matter in education policy.”

Part of the animus fueling these attacks is the hatred of the right-wing of unions. On top of that Teachers’ Unions have strongly supported liberal candidates and opposed conservative candidates. Anything that undermines teachers undermines Teachers Unions, it is all good to them.
LikeLike
yes, and in the states where we have no teachers’ unions, that is even more frustrating because we get it heaped on when we didn’t even have need for “union busting” (if in fact a need exists anywhere).
I don’t like the word “unrelenting” here—-I was thinking the parent protests and push back against the testing might have already caused some to side step their stance on VAM (kind of like Arne is now doing with CCSS). I guess not yet.
LikeLike
I heard a rather nebulous report on NPR this morning, citing Louisiana and Massachusetts for trying hard to do something about attracting quality teachers to poor areas. The implication, though, seemed that VAM was a good way to sort them out.
I never, ever knew public education was so political. Ever. Not until last year. Silly me.
The best analogy I can come up with for what we might be headed towards with CCSS and VAM is a Sunday School approach I was at first drawn to, and then grew to roll my eyes at. It’s called “Catechesis of the Good Shepherd” and it’s a Montessori approach to drawing children into their spiritual lives (Christian), formulated by Sophia Calvaletti. it has many, many good points. . .but in isolation it’s not enough. And that’s what I think CCSS and VAM will reveal themselves to be. . .not enough. Many Catholic churches use the Catechesis of the Good Shepherd and some Episcopal and Lutheran. A nyway, ii’s a very prescribed way of doing things in a Sunday School class (curriculum, set-up) and the teachers go through lots of training. But it stops there. It requires little to no innovation or creativity on the part of the teacher, and in my experience teachers get stuck in it and then resent being there because it’s very hard to recruit people to replace them (and in churches, it can be hard to recruit Sunday School teachers anyway). Or you end up with a dedicated following who make it their life cause. So you end up with little fiefdoms of overly-dedicated Catechists spinning their web to catch new teachers, and the burned out ones run (sometimes leaving the congregation all together). I wonder if that is what the prescribed methods of CCSS and VAM will bring about for teachers. I know some folks will not appreciate a church analogy, but it is groups of folks setting out to try something in regards to educating children (in fact, many states used to acknowledge training in Christian Ed for a teaching degree). The point here being that the less a teacher is bringing into the lessons themselves, the less soul it has about it (in my opinion). The caveat for an approach like this is so that a teacher won’t limit what a child can perceive by imposing their own views, connections, patterns, etc. But I think that assumption under-estimates the brains of most children. They can see beyond what a teacher is telling them. They know how to discern and sort out which adults have lives similiar to what they want their’s to be, etc.
I think Diane said it best when she said “we are fixing a problem we don’t have.” And the problem we do have is poverty.
LikeLike
xxxxxxxxxxxxxxxxxxxxxx I totally get what you’re saying from the Catholic (CCD) point of view, Joanna. My kids had a double-edged sword: a gifted public-school educator volunteer as their CCD teacher in those years [6th-7th-grade] when kids are beginning to think for themselves. They were able to see between what the old Boston Catechism demanded and the creative thinking required by their teacher… None of my kids made it to confirmation…
LikeLike
None of my three kids made it to a baptism!
Hell, my youngest was 11-12 years old before he set foot in a church (for my brother’s mother in law’s funeral).
LikeLike
So, then, does that mean kids will bail on school if they are simply pushed through a highly regulated system whereby teachers follow a script and get them ready for big tests? It’s worth thinking about. I’ve already heard very creative first graders tired of doing worksheets (which is what they often get stuck doing while teachers are “assessing” on I-pads all day).
Also, is there a conflict of interest when a curriculum director is on a task force (or something connected to) the company that makes the curriculum? Is that sort of thing common? I have seen that in counties where the county is requiring more assessment than the state is and the person requiring it happens to be connected to the company that makes the material. That doesn’t seem right to me.
LikeLike
One of the “nails” in the VAM coffin is a working paper by C. Kirabo Jackson, who essentially shreds the idea of test scores as a fair judge of teacher effectiveness. I wrote the first part of a summary about his paper on my blog: http://bltm.com/blog/2014/04/14/vamvomvom/ It makes for some interesting reading….
LikeLike
Robert,
Have you read Noel Wilson’s “Educational Standards and the Problem of Error” found at: http://epaa.asu.edu/ojs/article/view/577/700 ? If so, I’d appreciate any thoughts you may have about it!
Duane
Brief outline of Wilson’s “Educational Standards and the Problem of Error” and some comments of mine. (updated 6/24/13 per Wilson email)
1. A quality cannot be quantified. Quantity is a sub-category of quality. It is illogical to judge/assess a whole category by only a part (sub-category) of the whole. The assessment is, by definition, lacking in the sense that “assessments are always of multidimensional qualities. To quantify them as one dimensional quantities (numbers or grades) is to perpetuate a fundamental logical error” (per Wilson). The teaching and learning process falls in the logical realm of aesthetics/qualities of human interactions. In attempting to quantify educational standards and standardized testing we are lacking much information about said interactions.
2. A major epistemological mistake is that we attach, with great importance, the “score” of the student, not only onto the student but also, by extension, the teacher, school and district. Any description of a testing event is only a description of an interaction, that of the student and the testing device at a given time and place. The only correct logical thing that we can attempt to do is to describe that interaction (how accurately or not is a whole other story). That description cannot, by logical thought, be “assigned/attached” to the student as it cannot be a description of the student but the interaction. And this error is probably one of the most egregious “errors” that occur with standardized testing (and even the “grading” of students by a teacher).
3. Wilson identifies four “frames of reference” each with distinct assumptions (epistemological basis) about the assessment process from which the “assessor” views the interactions of the teaching and learning process: the Judge (think college professor who “knows” the students capabilities and grades them accordingly), the General Frame-think standardized testing that claims to have a “scientific” basis, the Specific Frame-think of learning by objective like computer based learning, getting a correct answer before moving on to the next screen, and the Responsive Frame-think of an apprenticeship in a trade or a medical residency program where the learner interacts with the “teacher” with constant feedback. Each category has its own sources of error and more error in the process is caused when the assessor confuses and conflates the categories.
4. Wilson elucidates the notion of “error”: “Error is predicated on a notion of perfection; to allocate error is to imply what is without error; to know error it is necessary to determine what is true. And what is true is determined by what we define as true, theoretically by the assumptions of our epistemology, practically by the events and non-events, the discourses and silences, the world of surfaces and their interactions and interpretations; in short, the practices that permeate the field. . . Error is the uncertainty dimension of the statement; error is the band within which chaos reigns, in which anything can happen. Error comprises all of those eventful circumstances which make the assessment statement less than perfectly precise, the measure less than perfectly accurate, the rank order less than perfectly stable, the standard and its measurement less than absolute, and the communication of its truth less than impeccable.”
In other word all the logical errors involved in the process render any conclusions invalid.
5. The test makers/psychometricians, through all sorts of mathematical machinations attempt to “prove” that these tests (based on standards) are valid-errorless or supposedly at least with minimal error [they aren’t]. Wilson turns the concept of validity on its head and focuses on just how invalid the machinations and the test and results are. He is an advocate for the test taker not the test maker. In doing so he identifies thirteen sources of “error”, any one of which renders the test making/giving/disseminating of results invalid. As a basic logical premise is that once something is shown to be invalid it is just that, invalid, and no amount of “fudging” by the psychometricians/test makers can alleviate that invalidity.
6. Having shown the invalidity, and therefore the unreliability, of the whole process Wilson concludes, rightly so, that any result/information gleaned from the process is “vain and illusory”. In other words start with an invalidity, end with an invalidity (except by sheer chance every once in a while, like a blind and anosmic squirrel who finds the occasional acorn, a result may be “true”) or to put in more mundane terms crap in-crap out.
7. And so what does this all mean? I’ll let Wilson have the second to last word: “So what does a test measure in our world? It measures what the person with the power to pay for the test says it measures. And the person who sets the test will name the test what the person who pays for the test wants the test to be named.”
In other words it measures “’something’ and we can specify some of the ‘errors’ in that ‘something’ but still don’t know [precisely] what the ‘something’ is.” The whole process harms many students as the social rewards for some are not available to others who “don’t make the grade (sic)” Should American public education have the function of sorting and separating students so that some may receive greater benefits than others, especially considering that the sorting and separating devices, educational standards and standardized testing, are so flawed not only in concept but in execution?
My answer is NO!!!!!
One final note with Wilson channeling Foucault and his concept of subjectivization:
“So the mark [grade/test score] becomes part of the story about yourself and with sufficient repetitions becomes true: true because those who know, those in authority, say it is true; true because the society in which you live legitimates this authority; true because your cultural habitus makes it difficult for you to perceive, conceive and integrate those aspects of your experience that contradict the story; true because in acting out your story, which now includes the mark and its meaning, the social truth that created it is confirmed; true because if your mark is high you are consistently rewarded, so that your voice becomes a voice of authority in the power-knowledge discourses that reproduce the structure that helped to produce you; true because if your mark is low your voice becomes muted and confirms your lower position in the social hierarchy; true finally because that success or failure confirms that mark that implicitly predicted the now self evident consequences. And so the circle is complete.”
In other words students “internalize” what those “marks” (grades/test scores) mean, and since the vast majority of the students have not developed the mental skills to counteract what the “authorities” say, they accept as “natural and normal” that “story/description” of them. Although paradoxical in a sense, the “I’m an “A” student” is almost as harmful as “I’m an ‘F’ student” in hindering students becoming independent, critical and free thinkers. And having independent, critical and free thinkers is a threat to the current socio-economic structure of society.
LikeLike
There’s a trunkful of studies from more than 15 years of research that show VAMs should NEVER be used to judge individual teachers, and some of these were written under USDE contracts. Go to the VAMBoozeled website if you want a list. There is another list for the 70% of teachers who don’t have statewide test scores to feed the VAM formulas. The whole policiy of judging teachers by the scores they produce is valid only if you think schools should be factories with production quotas for every work and the whole factory, then stack rank the techers and schools, then close the lowest performing schools and fire the lowest performing workers in the units left standing.
LikeLike
“Sanity is not statistical.”
― George Orwell, 1984
LikeLike
Most excellent!
😃
And from one of those old dead Greek guys:
“A good decision is based on knowledge and not on numbers.” [Plato]
But, but, but, snort and growl the proponents of data-drivel mismanagement, “you can’t manage what you can’t measure”!
Rheeally?
Not really…
W. Edwards Deming (aka “The Father of Quality”) who was not only a numbers/stats person to the bone but renowned for his successful application of statistical techniques to real world situations:
[start quote]
Manages are not faced with a deluge of information; they are faced with a deluge of figures. The challenge is to know when it is appropriate to respond to certain figures and when not to. Management must understand the theory of variation. If you don’t understand variation and how ti comes from the system itself, you can only react to every figure. The result is you often overcompensate, when it would have been better to just leave things alone.
[end quote]
But, they retort in with smug assurance, VAM and standardized test scores and graduation rates and drop-out rates and such tell us just about everything we need. Ok, there’s one or two minor things to add in, but c’mon!
[start quote]
He who would run his company on visible figures alone will in time have neither company nor figures. The most important figures for management (such as the multiplying effect on sales that comes from a happy customer, and the opposite effect from an unhappy customer) are either unknown or unknowable, but successful management must nevertheless take account of them.
[end quote]
VAM? Merit pay? How could they not help raise the quality of teaching and learning?
[start quote]
… Many companies in America have systems by which everyone in management or in research receives from his superiors a rating every year. On the basis of this rating, employees are ranked for raises—for example, outstanding high, outstanding,etc., on down to unsatisfactory. Management by fear would be a better name. … The basic fault of the annual appraisal is that it penalizes people for normal variation of a system.
[end quote]
[THE ESSENTIAL DEMING, 2013, Joyce C. Orsini, ed., pp. 170, 1-18, and 27, respectively]
it all adds up… [For those who are immersed in CCSS close[t] reading: A numbers/stat joke]
😎
LikeLike
KTA,
And as a follow up on your thoughts:
Doing the Wrong Thing Righter
The proliferation of educational assessments, evaluations and canned programs belongs in the category of what systems theorist Russ Ackoff describes as “doing the wrong thing righter. The righter we do the wrong thing,” he explains, “the wronger we become. When we make a mistake doing the wrong thing and correct it, we become wronger. When we make a mistake doing the right thing and correct it, we become righter. Therefore, it is better to do the right thing wrong than the wrong thing right.”
Our current neglect of instructional issues are the result of assessment policies that waste resources to do the wrong things, e.g., canned curriculum and standardized testing, right. Instructional central planning and student control doesn’t – can’t – work. But, that never stops people trying.
The result is that each effort to control the uncontrollable does further damage, provoking more efforts to get things in order. So the function of management/administration becomes control rather than creation of resources. When Peter Drucker lamented that so much of management consists in making it difficult for people to work, he meant it literally. Inherent in obsessive command and control is the assumption that human beings can’t be trusted on their own to do what’s needed. Hierarchy and tight supervision are required to tell them what to do. So, fear-driven, hierarchical organizations turn people into untrustworthy opportunists. Doing the right thing instructionally requires less centralized assessment, less emphasis on evaluation and less fussy interference, not more. The way to improve controls is to eliminate most and reduce all.
Former Green Beret Master Sergeant Donald Duncan (Viet Nam) did when he noted in Sir! No Sir! that:
“I was doing it right but I wasn’t doing right.”
And from one of America’s premier writers:
“The mass of men [and women] serves the state [education powers that be] thus, not as men mainly, but as machines, with their bodies. They are the standing army, and the militia, jailors, constables, posse comitatus, [administrators and teachers], etc. In most cases there is no free exercise whatever of the judgment or of the moral sense; but they put themselves on a level with wood and earth and stones; and wooden men can perhaps be manufactured that will serve the purpose as well. Such command no more respect than men of straw or a lump of dirt.”- Henry David Thoreau [1817-1862], American author and philosopher
LikeLike
See John Kuhn’s great new book Fear and Learning in America: Bad Data, Good Teachers, and the Attack on Public Education. — Edd Doerr (arlinc.org)
LikeLike
“The point here being that the less a teacher is bringing into the lessons themselves, the less soul it has about it (in my opinion).”
Exactly, Joanne. When everything you do is prescribed by the authorities, any monkey can do it and that is exactly what they want. And if they can figure out how to measure all those soft skills that make us all individuals (however bogus, apparently, if VAM is anything to judge by), then what need do they have at all for living breathing human beings as teachers?
I used to subscribe to the apolitical school of thought. It made me an easy target.
LikeLike
JoannA. 🙂
LikeLike
The ASA just slammed VAM! Read Valerie Strauss’s article here:
http://www.washingtonpost.com/blogs/answer-sheet/wp/2014/04/13/statisticians-slam-popular-teacher-evaluation-method/
LikeLike