Remember that Arne Duncan said that there was too much testing, that testing was sucking the oxygen and joy out of classrooms? New York didn’t get the message. In that state, state tests count for 20% of educator evaluations, and local assessments count for another 20%. That is the agreement negotiated with the unions when the state won Race to the Top funding.
That was then, this is now.
The Néw York Board of Regents want state test scores to count for 40% of the evaluations of teachers and principals. This report was was confirmed to me by someone in Albany.
It matters not to the Regents that test-based evaluation is not working anywhere else. It matters not that the AERA and the National Academy of Education warned against it, warned that it would incentivize teachers to avoid high-needs students. It matters not that the American Statistical Association warned against using test scores to rate individual teachers since they affect only 1-14% of variation in student scores.
The ASA said: “Attaching too much importance to a single item of quantitative information is counter-
productive—in fact, it can be detrimental to the goal of improving quality. In particular, making changes in response to aspects of quantitative information that are actually random variation can increase the overall variability of the system.”
Unlike the state of Vermont, which refuses to rate teachers and principals by test scores, Néw York’s Regents will plunge ahead, regardless of the damage they do to teachers, principals, students, and communities.
Diane Ravitch's blog 
If I facepalmed as hard as something the egregious called for me to, I would require hospitalization.
My take is they can’t handle, nor want to handle the 20% – written ELA Performance-based-assessments. It is easier for the state to simply look at state test scores and proceed on the BS 40% rating. With children being labeled 70% failures on the Common Core exams, teachers should want very little to do with being evaluated on these poorly constructed/rushed state exams.
NYSED is not only incompetent, they are lazy, too.
Let us not forget all the legislators that recently voted to reappoint incumbent Regents, despite the outcry from educators across the state. Remember to vote, it counts more than ever.
Good point, here is the voting breakdown-
http://www.nysape.org/regents-vote-scorecard.html
So I was under the impression Cuomo said teachers would not be evaluated by scores?? So which is it, or is Cuomo just trying to get votes? And where is NYSUT and the UFT response which I am sure will be double-talked spin?
Just the Common Core assessments, due to the failed implementation.
Of course, using the old state exams while everyone has a gun to the head to demonstrably implement CC standards is equally ridiculous, BESIDES the overarching fallacy of relying on test scores as any sort of final word on anything!
Cuomo said that if a teacher is rated “developing” or “ineffective” based on state (CC) math or ELA tests scores that their HEDI score will be recalculated using a different, but equally convoluted form of voodoo math.
This is the official statement fro the NYSED website:
Chapter xx of the Laws of 2014 (Governor’s Program Bill No. 56) applies to classroom teachers or building principals rated as Ineffective or Developing on their composite ratings in the 2013-14 and/or 2014-15 school years in an APPR conducted pursuant to Education Law §3012-c, where some portion of their APPR rating was based on a State assessment aligned with the Common Core in English language arts and/or math in grades 3-8. For these teachers and principals, Chapter xx of the Laws of 2014 provides that districts and BOCES must provide a “safety net calculation”. If the “safety net calculation” is higher than the rating under Education Law §3012-c, the district/BOCES is precluded from making certain employment decisions based on the teacher’s or principal’s rating under Education Law §3012-c (termination decisions, tenure decisions, expedited hearings pursuant to Education Law §3020-a and decisions related to retention). Provided, however, that nothing in Chapter xx of the Laws of 2014 shall prevent the use of the following for employment decisions: observations, local assessments or other measures of the performance of the teacher or principal, other than their rating or a state 3-8 assessment aligned with the Common Core, whether or not they were included in an APPR.
Can someone remind me what the “local assessments” are in NYC?
Local assessments are whatever measure a district decides to use locally for the 20 percent local measure of a teacher’s evaluation. teachers in grades 3-5 use standardized test scores for the tested portion and the “local” portion of their scores. Some buildings use locally designed tests, programs like Iready, and even PSAT scores for the local 20 percent. Some use an average measure of proficiency for all final exams. the point is, they are locally controlled and locally agreed upon. This is partly about taking away local control and partly Cuomo trying to make New York the new Florida? And if you think NYSED and Tisch do anything without Cuomo’s blessing, think again.
In some elementary schools they use the MOSL…measures of student learning. It is a test developed by NYC given at the beginning of the year for k-5. At the end of the year k-2 take another MOSL…3-5 are evaluated on standardized test scores. The MOSL does not resemble the state test, and for kindergarten students to test with this is ridiculous. Teachers are pulled from their classes to mark the MOSL over several periods/days.
New York Board of Regents is Bunch of Rodents without the teeth. Or we might say, they are Bunch of Robots.
Teachers if disabled students will never be considered “highly effective” based on this system!
Exactly, Laurie, and who will want to become a special education teacher? Who will want students with disabilities in their classrooms? To expect children whose disabilities have already slowed their progress to close a multi-year gap in one year places an impossible burden on the student and the teacher. Too much time is already lost due to the insistence that CWD spend hours “exposed” to grade level instruction without the tools, staff, or opportunity for modification and adaption of engageny modules to allow the child to access the material. This pushes out the appropriate foundational or life skills teaching the student needs much more.
Don’t know what they’re smoking in Albany, but I want some!
No, no, Glenn, you’ll become one of Them, a talking points, buzz phrase-spouting zombie.
The goals are to reduce pension liabilities and transform teachers into temps.
YUP
You got it!
In 2013, 15 of the 19 states that received RttT funds had an evaluation system wherein 50% of every teacher’s rating is determined by the growth scores of their students. Looks like NY at 40% is a lark. Just kidding.
What is a growth score? In RttT legislation, “Student growth means the change in student achievement for an individual student between two or more points in time.” “Student achievement means (a) For tested grades and subjects: (1) A student’s score on the State’s assessments under the ESEA; and, as appropriate, (2) other measures of student learning, such as those described in paragraph (b) of this definition, provided they are rigorous and comparable across classrooms. (b) For non-tested grades and subjects: Alternative measures of student learning and performance such as student scores on pre-tests and end-of-course tests; student performance on English language proficiency assessments; and other measures of student achievement that are rigorous and comparable across classrooms”
This accountability imperative is just the beginning. The federal definition of an “effective” teacher requires attention to the rates at which student’s scores increase. “Effective teacher means a teacher whose students achieve acceptable rates (e.g., at least one grade level in an academic year) of student growth (as defined in this notice). “Highly effective teacher means a teacher whose students achieve high rates (e.g., one and one-half grade levels in an academic year) of student growth (as defined in this notice” (Federal Register / Vol. 74, No. 221 / Wednesday, November 18, 2009 / Rules and Regulations Department of Education, 34 CFR Subtitle B, Chapter II [Docket ID ED–2009–OESE–0006] Final Definitions 59804-59811).
If should be obvious that calculations to determine “rates” of growth depend on a data system that matches the test scores of individual students and the “teacher of record” for a given student and test. These records serve as “baselines” for estimates of the “value-added” by a teacher to the scores of their students and various sub-groups. VAM produce these estimates for teachers whose job-assignments are directly linked to statewide tests.
SLOs are a proxy for VAM for the 70% of teachers in so-called nontested subjects. This proxy is intended to make the scores for rating all teachers comparable or nearly so. Since the Denver pilot in 1999, promoters of SLOs have marketed this convoluted version of mangement-by-objectives as “adding a little science to the art of teaching.” Nonsense.
The categories and criteria for a typical SLO forward the illusion that every step in the process is scientific. Thus, students in classes are dubbed a “population.” Records from prior grades become “baseline data” for profiling and grouping students. “Expected growth” is a prediction of posttest scores, but stripped free of any theory that might leverage reasoning about the expected outcomes. “Growth” is a euphemism for an expected increase in scores, pretest to posttest.
In effect, SLOs are framed and rated as if the teacher is documenting a one-group pretest-posttest experiment for the population named in the SLO with no control group, and typically with an arbitrary demand for multiple instructional strategies and measures.
In a typical SLO process, raters look for evidence of that a teacher sets a “high quality” and “rigorous” learning target, defined as aiming for “more than a one-year gain in proficiency.” Achieving that outcome corresponds to the federal definition for a highly effective teacher. The precise definition of “more than a one-year gain in proficiency” is: “A teacher’s students will exceed the average rate of increase in scores produced by other teachers who have a job-alike assignment” (Fed. Reg., 2009). This definition also means that the evaluation system is designed to produce ratings from data arrayed with “stretch,” much like a bell curve.
The illusion of scientific precision is conveyed, but these metrics are decoupled from reasoning about what proficiency is for a specific domain of study or task, and for given individual or group. The federal definition also suppresses attention to how and when a given proficiency may be attained, and why it may educationally significant .
Given all of these variables and criteria, no reliable and valid inferences can be made about the effect of the teacher on the posttest scores. In this respect, the use of SLOs to justify judgments about a teacher’s effectiveness is not only blatantly unscientific but also unethical.
The Denver template for writing an SLO has variants in other states, including New York, Ohio, Indiana, Rhode Island, Louisiana.
Promoters promoters of SLOs and VAM seem to assume that for every subject, every grade level, and every standard there also exists a deep reservoir of teaching methods differentiated for groups and individual students—all backed by current empirical research and valid, reliable tests.
These assumptions are without any foundation in fact. The practice of requiring teachers to write measurable student objectives is not entirely new, but it has been fashioned into an instrument to support test-driven teaching, an unparalleled effort to micro-manage and standardize education, and install pay-for-performance as the national norm for teacher compensation.
At least 27 states are now requiring all teachers, not just those in the so-called nontested subjects, to comply with a pseudo-scientific writing exercise that includes “measurable targets for learning,” predictions about the success of the teacher and students in meeting them. This process (rejected by savvy CEOs) ends by rating teachers on whether they are good predictors of their students’ scores on the SLO tests.
In Austin Texas, one of the early adopters of this method of evaluation in connection with pay for performance (the aim of RttT), researchers found that teachers with several years of training in SLO writing learned to select content and objectives and targets likely to make them eligible for a bonus. In other words, they learned to teach to the SLO tests.
Pay-for-performance is not the norm in comparable professions. Successful CEOs no longer use management-by-objectives, they seek improvements in the workplaces of their employees. Pay for performance invites cheating and other strategies for gaming the system. Federal policies and states that are following this model of teacher evaluation–the most recent Maryland–have no credible peer reviewed research to warrant SLOs or VAMs…not my opinion but documented in four recent reports from the Institute of Educational Sciences. There is no there there for “evidence.”
These measures, and the mandates that have made them central in teacher evaluation, are strictly ideological and hostile to the very idea of professionalism in education.
Laura,
Excellent summary. Thanks for putting that together!
Have you read Wilson’s “Educational Standards and the Problem of Error”? If so, your thoughts on it.
Thanks,
Duane
Yup, that’s it: “these metrics are decoupled from reasoning” !
Laura, my husbands SLOs and Artifacts are a work of art.
Ultimately, a year’s growth will result with the same score from year to year, whether a 1, 2, 3, or 4. It is the rare child who leapfrogs into a new “category”. Our IQ doesn’t automatically change with each passing year.
And then there is the actual exam. Unless it is age appropriate and reflects a student’s true abilities, the exams, and thus the scores, are irrelevant.
What then does an individual score prove?
This effort is part of the “Teaching is the Core” plan from NYSED. Basically the plan is to get local districts to end using SLO pre/post testing to evaluate teachers without a state test tied to their course. Districts are getting grant money if they do this. NYSED will be using this plan to shift the blame for increased testing to local districts and unions and away from Albany where the problem started.
The SLO process is laughable but at least teachers and schools have local control over the tests and scores as opposed to turning over 40% to the state test results and the voodoo math used to create teacher (and student) scores from NYSED.
Either way our kids lose.
Here’s a question that I hope some one can answer for me: Because it is well-established that factors such as parental discord, hidden health issues, household financial problems, and myriad behavioral factors can heavily influence a student’s achievement in school and performance on standardized tests, then in accord with the Due Process rights that teachers have, can a teacher whose job is on the line because of students’ poor test scores request detailed information about students’ home life, parental relations, family finances, and other pertinent information? If so, that would raise such a political backlash against using test scores to evaluate teachers that use of test scores to evaluate teachers would be swiftly dumped.
Local assessment scores can be inflated because teachers can use rubrics they have created-often these rubrics make it easy to score well and difficult to do poorly on. Teachers score local assessments of their co-workers, and they are reluctant to give bad scores to the students of these co-workers.
Another problem is that administrators are hesitant to give perfect or high scores to teachers, fearing too many high scores will trigger state audits. (In NY, admins scores are 20 percent of a teacher’s overall rating)
The system is a total waste of time and resources! All of these scientific and management types should stop the BS and go spend a few decades in a classroom.
“It matters not to the Regents that test-based evaluation is not working anywhere else. It matters not that the AERA and the National Academy of Education warned against it, warned that it would incentivize teachers to avoid high-needs students. It matters not that the American Statistical Association warned against using test scores to rate individual teachers since they affect only 1-14% of variation in student scores.”
I matters not that Noel Wilson completely destroys the ontological and epistemological bases for educational standards and standardized testing so that any results gleaned from those invalid educational malpractices are, as he concludes, “VAIN AND ILLUSORY” and I add; any usage “COMPLETELY UNETHICAL”.
To understand why, read and comprehend what Wilson has proven in his never refuted nor rebutted “Educational Standards and the Problem of Error” found at: http://epaa.asu.edu/ojs/article/view/577/700
Brief outline of Wilson’s “Educational Standards and the Problem of Error” and some comments of mine. (updated 6/24/13 per Wilson email)
1. A description of a quality can only be partially quantified. Quantity is almost always a very small aspect of quality. It is illogical to judge/assess a whole category only by a part of the whole. The assessment is, by definition, lacking in the sense that “assessments are always of multidimensional qualities. To quantify them as unidimensional quantities (numbers or grades) is to perpetuate a fundamental logical error” (per Wilson). The teaching and learning process falls in the logical realm of aesthetics/qualities of human interactions. In attempting to quantify educational standards and standardized testing the descriptive information about said interactions is inadequate, insufficient and inferior to the point of invalidity and unacceptability.
2. A major epistemological mistake is that we attach, with great importance, the “score” of the student, not only onto the student but also, by extension, the teacher, school and district. Any description of a testing event is only a description of an interaction, that of the student and the testing device at a given time and place. The only correct logical thing that we can attempt to do is to describe that interaction (how accurately or not is a whole other story). That description cannot, by logical thought, be “assigned/attached” to the student as it cannot be a description of the student but the interaction. And this error is probably one of the most egregious “errors” that occur with standardized testing (and even the “grading” of students by a teacher).
3. Wilson identifies four “frames of reference” each with distinct assumptions (epistemological basis) about the assessment process from which the “assessor” views the interactions of the teaching and learning process: the Judge (think college professor who “knows” the students capabilities and grades them accordingly), the General Frame-think standardized testing that claims to have a “scientific” basis, the Specific Frame-think of learning by objective like computer based learning, getting a correct answer before moving on to the next screen, and the Responsive Frame-think of an apprenticeship in a trade or a medical residency program where the learner interacts with the “teacher” with constant feedback. Each category has its own sources of error and more error in the process is caused when the assessor confuses and conflates the categories.
4. Wilson elucidates the notion of “error”: “Error is predicated on a notion of perfection; to allocate error is to imply what is without error; to know error it is necessary to determine what is true. And what is true is determined by what we define as true, theoretically by the assumptions of our epistemology, practically by the events and non-events, the discourses and silences, the world of surfaces and their interactions and interpretations; in short, the practices that permeate the field. . . Error is the uncertainty dimension of the statement; error is the band within which chaos reigns, in which anything can happen. Error comprises all of those eventful circumstances which make the assessment statement less than perfectly precise, the measure less than perfectly accurate, the rank order less than perfectly stable, the standard and its measurement less than absolute, and the communication of its truth less than impeccable.”
In other word all the logical errors involved in the process render any conclusions invalid.
5. The test makers/psychometricians, through all sorts of mathematical machinations attempt to “prove” that these tests (based on standards) are valid-errorless or supposedly at least with minimal error [they aren’t]. Wilson turns the concept of validity on its head and focuses on just how invalid the machinations and the test and results are. He is an advocate for the test taker not the test maker. In doing so he identifies thirteen sources of “error”, any one of which renders the test making/giving/disseminating of results invalid. And a basic logical premise is that once something is shown to be invalid it is just that, invalid, and no amount of “fudging” by the psychometricians/test makers can alleviate that invalidity.
6. Having shown the invalidity, and therefore the unreliability, of the whole process Wilson concludes, rightly so, that any result/information gleaned from the process is “vain and illusory”. In other words start with an invalidity, end with an invalidity (except by sheer chance every once in a while, like a blind and anosmic squirrel who finds the occasional acorn, a result may be “true”) or to put in more mundane terms crap in-crap out.
7. And so what does this all mean? I’ll let Wilson have the second to last word: “So what does a test measure in our world? It measures what the person with the power to pay for the test says it measures. And the person who sets the test will name the test what the person who pays for the test wants the test to be named.”
In other words it attempts to measure “’something’ and we can specify some of the ‘errors’ in that ‘something’ but still don’t know [precisely] what the ‘something’ is.” The whole process harms many students as the social rewards for some are not available to others who “don’t make the grade (sic)” Should American public education have the function of sorting and separating students so that some may receive greater benefits than others, especially considering that the sorting and separating devices, educational standards and standardized testing, are so flawed not only in concept but in execution?
My answer is NO!!!!!
One final note with Wilson channeling Foucault and his concept of subjectivization:
“So the mark [grade/test score] becomes part of the story about yourself and with sufficient repetitions becomes true: true because those who know, those in authority, say it is true; true because the society in which you live legitimates this authority; true because your cultural habitus makes it difficult for you to perceive, conceive and integrate those aspects of your experience that contradict the story; true because in acting out your story, which now includes the mark and its meaning, the social truth that created it is confirmed; true because if your mark is high you are consistently rewarded, so that your voice becomes a voice of authority in the power-knowledge discourses that reproduce the structure that helped to produce you; true because if your mark is low your voice becomes muted and confirms your lower position in the social hierarchy; true finally because that success or failure confirms that mark that implicitly predicted the now self evident consequences. And so the circle is complete.”
In other words students “internalize” what those “marks” (grades/test scores) mean, and since the vast majority of the students have not developed the mental skills to counteract what the “authorities” say, they accept as “natural and normal” that “story/description” of them. Although paradoxical in a sense, the “I’m an “A” student” is almost as harmful as “I’m an ‘F’ student” in hindering students becoming independent, critical and free thinkers. And having independent, critical and free thinkers is a threat to the current socio-economic structure of society.
By Duane E. Swacker
It matters, not I matter, ay ay ay!
Duane, being an A student never held me back. Instead it gave me the confidence to move forward. Plus I learned the value of hard work because my As were earned, not handed to me. Success breeds success. And failure gives you the opportunity of a do over. The goal is to learn from our mistakes. A side effect is a better understanding of those who also strive despite set backs.
Isn’t this the American Way – boot straps and all?
“Success breeds success. And failure gives you the opportunity of a do over.”
Ellen, was that written with tongue in cheek?
Of course, Wilson is more relevant in the current system where the majority are destined to fail. Hard to justify institutionalized “Fs”.
No amount of evidence, statistical data or research is going to be understood, much less change the minds, of the educational leaders driving the wagon in the state. This is a conceptual and literacy deficiency issue compounded by ideology, the “big data driven” management fad that did not work in the business world (where metrics are exceedingly simple compared to education), sheer stupidity and by economic forces that benefit from all this nonsense. I won’t even question the intentions of many – we need not. From King downwards they simply do not have the statistical or research knowledge to follow what a stupid mistake this data driven decision making bandwagon is all about. Carmen has a different feel for education – and is rightfully skeptical of it but she inherited a machinery that has a huge economic stake in the apparatus of testing and management, and the ship left port long ago and its going to take enormous efforts to change the momentum left by the technocrats before her. And NY is not the only state suffering from high level numerical/statistical/empirical incompetence – Puerto Rico, and I am sure many other states are in the same situation. Part of the problem is that the leadership cannot really deeply understand the limits of educational measurements, nor understands the enormous data quality problems, or the statistical and experimental constraints under which education operates…and they are lured by the power of numbers and data crunching…which they really don’t understand in the first place…
What the commissioner or the Regents want is irrelevant – the law, which is highly unlikely to change establishes 60-20-20 values – 60% supervisory judgment based upon an agreed upon rubric, 20% locally negotiated and 20% student growth scores for teachers in grades 3-8.
Both the commissioner and the Regents are held in low regard by the legislature – and electeds have no reason to do battle with the unions over changing a controversial law.
And, while I hold Diane in extremely high regard, the members of the Regents with whom I interact are baffled by the suggestion …
The legislature was happy enough with the Regents to reauthorize most of them in their positions a few months ago, despite the fact that there was substantial outcry from parents and teachers throughout the state to replace them.
Teachout : 9/9/14
Let’s do this and get those clowns out of there.
Yes, please, everyone vote for Zephyr Teachout in the upcoming primary.
We must do everything we can to get Bill D. out from under Cuomo’s thumb.
Did you see the NY Times officially refused to endorse Cuomo, or Teachou, It is a very subtle yet de-facto endorsement for Teachout if you read between the lines.
Cuomo refuses to debate. He has nothing to gain and everything to lose. He should still be called out for ignoring this important and traditional campaign practice. A moral victory if Teachout can get at least 25% of the vote. GO Zephyr.
The NY Times ed board is explicitly encouraging people to vote for Teachout in the primary to send a message to Cuomo.
Also, Zephyr nails Cuomo on his ties to Christie in the Daily News:
http://m.nydailynews.com/opinion/cuomo-enabled-christie-tunnelgate-article-1.1917992
The Nation endorses Teachout.
http://m.thenation.com/article/181379-zephyr-teachout-governor-new-york
I can’t wait to vote for her. I met her this past weekend and she is even more incredible, in person.
With the opening of school, I hope all NY teachers aware of her educational policies, go to her website, and print and hang a campaign poster for their faculty room. Those who are aware, need to share the knowledge that there is a pro-teacher alternative and voting in the primary is essential. And with all the details one needs to remember at the beginning of a new school year, it is definitely helpful to remind others, the primary is 9/9/14!
Another newspaper is endorsing her, basically for the primary
9/9/14
http://m.rochestercitynewspaper.com/rochester/to-protest-corruption-teachout-for-governor/Content?oid=2426831
Reblogged this on seldurio and commented:
We MUST fight against this!
Most teachers do not teach subjects tested by the state. Grades 3 to 8 math and ELA teachers and high school Regents teachers. To tie ALL teachers in NYS to a “state test” will require a contortionist. It will open an educational Pandora’s Box. And it will open a floodgate of litigation.
So, you’re an 8th grade music/band teacher and now 40% of your evaluation is “tied” to the math and ELA scores in your building. Now what? Ignore this idiocy and just teach the best MUSIC program you can. Or become an educational contortionist as you helplessly try to infuse CC math and ELA into your music program. This may be the STUPIDIST idea yet from NYSED.
Correction: This may be the stupidest idea yet from the [Board of Regents].
I a stupid contest between NYSED and the BOR, it would probably be close.
NY Teacher: Florida figured out how to rate the 70% of teachers who teach non-tested subjects. They are assigned the scores of their school, or they choose math or English. Chime in, Florida teachers.
That’s an awful lot of pressure on math and ELA teachers. Wondering how it affects all other teachers in Florida?
What about HS teachers? CC tests have been limited to grades 3 to 8.
The attacks will continue until teachers decide to fight back by walking out of the schools en masse.
“Only a fool would let his enemy teach his children.” – Malcolm X
In the name of fixing the American Educational System, it is being destroyed by those who know nothing about education. A national crisis will occur if they continue on this path because very few will enter into the educational system to become teachers.