This is the executive summary of the statement of the American Statistical Association on the use of value-added assessment to evaluate teachers. Please share it with other teachers, with principals, and school board members. Please share it with your legislators and other elected officials. Send it to your local news outlets. The words are clear: Teachers account for between 1 and 14% of the variation in test scores. And this is very important to remember: “Ranking teachers by their VAM scores can have unintended consequences that reduce quality.”
ASA Statement on Using Value-Added Models for Educational Assessment
April 8, 2014
Executive Summary
Many states and school districts have adopted Value-Added Models (VAMs) as part of educational accountability systems. The goal of these models, which are also referred to as Value-Added Assessment (VAA) Models, is to estimate effects of individual teachers or schools on student achievement while accounting for differences in student background. VAMs are increasingly promoted or mandated as a component in high-stakes decisions such as determining compensation, evaluating and ranking teachers, hiring or dismissing teachers, awarding tenure, and closing schools.
The American Statistical Association (ASA) makes the following recommendations regarding the use of VAMs:
- The ASA endorses wise use of data, statistical models, and designed experiments for improving the quality of education.
- VAMs are complex statistical models, and high-level statistical expertise is needed to develop the models and interpret their results.
- Estimates from VAMs should always be accompanied by measures of precision and a discussion of the assumptions and possible limitations of the model. These limitations are particularly relevant if VAMs are used for high-stakes purposes.
o VAMs are generally based on standardized test scores, and do not directly measure potential teacher contributions toward other student outcomes.
o VAMs typically measure correlation, not causation: Effects – positive or negative – attributed to a teacher may actually be caused by other factors that are not captured in the model.
o Under some conditions, VAM scores and rankings can change substantially when a different model or test is used, and a thorough analysis should be undertaken to evaluate the sensitivity of estimates to different models.
• VAMs should be viewed within the context of quality improvement, which distinguishes aspects of quality that can be attributed to the system from those that can be attributed to individual teachers, teacher preparation programs, or schools. Most VAM studies find that teachers account for about 1% to 14% of the variability in test scores, and that the majority of opportunities for quality improvement are found in the system-level conditions. Ranking teachers by their VAM scores can have unintended consequences that reduce quality.
Diane Ravitch's blog 
Blaming the worker is always easier than fixing a broken system.
This was a message that W. Edwards Deming championed in his effort to hep struggling American industrial companies meet the challenges of a growing competitive marketplace.
He often used the Red Bead Experiment (The New Economics, W Edward Deming p 158) to illustrate the how powerful the the system within which we work is in defining outcomes and how it limits any individual’s ability to improve results.
Wouldn’t it wonderful to have PResident Obama, Secretary Duncan, Bill Gates and the other leaders of what is called reform participate in this exercise?
But I fear, as the predecessors who lead GM, Ford and our other industrial giants into the ground, they will continue to ignore knowledge and exile real expertise – as Deming was forced to work in Japan.
Martin Levine Levine Partners, LLP
I look at it little different VAM has freed me.
I wanted to thank the state of Florida for inspiring me to be mediocre, my principal and students and their their parents may not be so grateful. You see senate bill 736 which ties teacher evaluations to teacher pay is in full effect this year. When the state legislature passed the bill they said they would be rewarding the best teachers but the truth is they have inspired most of the best teachers, those with experience and a proven track record to just get by.
As a veteran teacher with a professional contract, one I have to renew every five years, I am guaranteed a job as long as I prove to be effective and don’t commit any acts of moral turpitude. In short I can’t be fired just because my principal doesn’t like me for refusing to work prom, leaving before six, he has a neighbors cousin who wants to try the job or some other reason. I am also not eligible for merit pay.
Now I could give all of above up and go on a one year contract where I could be fired for any reason at the end of the year but I don’t think that’s a good idea and most veteran teachers agree with me.
Now here’s the rub, I only have to score an 80 on a 200 point scale to be considered effective something I could probably do sleep walking through the day in my pajamas. To give you some scale last year I received 77 points in student growth, which means I only needed three points from my principal (I got sixty), now it’s true this year I have to depend on the district average for reading teachers, but as things stand now that’s just 20 points.
Segue, half of my evaluation is based on a subject I don’t, teaching students I don’t know at schools I don’t work at. Welcome to Florida where up is down, black is white and people who want to dismantle public education are in charge.
My career has been turned into a game of get to 80 points, because at 80 I am effective and I will get paid the same as I would have had I gotten to 200, not that anybody in my district, who has to depend on the averages of a group of teachers to get their student growth scores is going to get to close to 200 anyway.
Lesson plans? What’s the point I can only lose ten points if I don’t do them.
Professional development? Nope I can free up my afternoons now because the most I can lose for that is another ten too.
Using Data and creating a nurturing environment are now optional as well.
It might be difficult to half ass the rest of my career but every time I feel bad for not giving my all to my students I can just think about the disrespect the state of Florida has heaped upon me and all the other professional teachers.
There are other small problems too, like how merit pay has very little evidence that says it works and tons that says it doesn’t and the states use of VAM scores that don’t factor in poverty and the Department of Education says are wrong more than a third of the time but to be honest what does it matter to me. I have just effectively toiled for the last 14 years for the honor of having to just score an eighty on my evaluation for the rest of it.
So Florida, thanks, not for the opportunity to earn more money but the clarity to know from now on I come first.
Excellent comment Chris. As a fellow Florida teacher, I share your frustration with those in charge in Tallahassee. I just left the classroom when I won a seat on our county’s school board. Change has to start somewhere. When the local grassroots can affect our local legislators. But with the corporatists in charge, and those who support public education, not showing up to vote in the governor’s race… it’s an uphill slog.
Same here in Dallas! We now have a performance pay system where very few teachers will get any raise at all–much less a sizable one.
Test scores are a huge chunk of the system and, with low-income, transient students, teachers realize we’re pretty much going to strike out on that domain.
So teachers have quickly figured out the minimum amount of paperwork that needs to be done to keep our jobs and that’s all that will be done for the rest of the domains.
Committees? You have to head up one or two and document the impact of your Teachers Against Pollution or whatever committee to get a raise. So why bother?
There’s a freaking rubric for everything, so teachers scan down to the lowest acceptable level on the rubric and ignore the requirements for the higher levels.
Going to see the school play or attending one of the kids’ games, which used to be deeply appreciated by admins, no longer matters. You need to attend multiple events a week to be counted among the “dedicated” who are worthy of a 1% raise.
For the first time in years, my school does not (and will not) have 100% of the teachers paying the $10 PTA dues because I think you have to be a member to get the pay raise that you’re not going to get anyway.
Hi Chris,
I’ve taught HS science for 23 yrs in Miami-Dade. Here is a letter I wrote and sent to our school board.]
————————————————
The Golden Rule of Accountabilty:
VAM unto others as you would have others VAM unto you
To: School, District and State Educational Administrators
From: Your Passionate, Professional and Committed Teachers
Re: VAM (Value Added Models, or Very Arbitrary Measures)
Dear Administrators and Pedagogical Theoreticians:
Since you believe teachers should be held accountable for results and have a proportion of their paycheck tied to test results, then you must also believe that you too should have part of your paycheck determined by test results. If you disagree with this then tell us why; give us a rationale for your hypocrisy.
Teachers complain that VAM statistics ignore some basic variables (ex. socioeconomic levels) and that they will be blamed for poor performance (aka: low test scores) for factors that are beyond the control and scope of their classrooms and influence. They will even point out all the variables that lie within the affective domain (desire, perseverance, work-ethic, love to learn, will to self-actualize [self-efficacy]) are primarily developed within the domain of the family (after all, family support-input is the primary predictor of success in school and post-secondary pursuits). Yet, under VAM teachers will get the blame for low performance, even though the equations are supposed to consider primary factors (but ignore socioeconomics and family). So even when we do our best some kids will not learn in spite of our efforts, though our paychecks will suffer.
So, then you too dear administrators must get blamed for events beyond your control, because if “it takes a village to raise a child”, then all in the village should suffer the negative reinforcement (lower paycheck) when the child fails.
We, teachers, may decide to blame you for your possibly deficient leadership or ineffective mentoring, or buying junky curriculum, or…? If, in the metaphor of the learning-village, part of our effectiveness as teachers is tied to your effectiveness as leaders and guides, then you too should be held accountable!
Are VAM (Value-Added Model, or Very Arbitrary Measure) truly unbiased and take into account as many covariates, or confounding variables (and factor these out), so that the “signal to noise ratio” is significant. One analysis by the American Statistical Association showed that only 1-14% of VAM data variability was affected by teacher input, the other 86% to non-teacher factors. So, the signal-to-noise ratio of most districts’ VAMs are 86% noise, and at best 14% signal (effects “created” by teacher input).
Yet, if this is true, then should not next year’s merit pay schedule only show, at most, a 14% salary gain for high VAM values? Why should salaries change more than 14% if teacher influence on test scores is less than or equal to that value?
By the way, I have asked UTD and MDCPS (my principal) when and where the 2 salary schedules (PSC, CC or Merit Pay?) for next year, when all of SB 736 is to be implemented, yet nobody knows where to find them. Is MDCPS working hard and fast to get these published, so that teachers can see their choices for next year? I though by August 2014 the full Merit Pay choice is supposed to be implemented, but I hear nothing from the district about this.
If VAMs don’t consider socio-economic levels, or past or current family life, then they are biased and will ignore the work of those who teach lower-level students, or may be handicapped against those who teach upper-level students. For example, is the value of helping out a low income, inner-city, child go from a 1 to a 2 in the FCAT, the same as that teaching in a rich suburb and having a score go from a 3 to a 4? Intuition would say no; that it might take more work (and therefore more value?) to help out the student who grew up in a less-fortunate environments.
Will VAM statistics factor this into account? If so, how, and if not then they are seriously flawed. Should VAMs be handicapped, so that gains made in the lower 1/3 of the student population are weighted more than the upper 1/3, because it takes more teacher-effort to raise the lower 1/3? If little or no gains are made by the upper 1/3 is it because they have poor teachers, or that it is more difficult to create gains with groups already in the 80-90 percentiles? I don’t believe VAM equations look at these variables, but then the real mystery is that most of the equations districts use are hidden and cryptic, only found in research journals that analyze and comment upon them, but not out in the open for all teachers to see.
Several times I’ve asked the UTD and my own principal to show me where at the MDCPS website is our VAM posted, but have never received an accurate answer. One would expect that at the Teacher Portal there would be a big bright banner stating “MDCPS VAM Equation” because I thought stakeholders are valued to the degree they are informed about decisions that affect them. Yet, when it comes to VAM I guess the district’s policy is the inverse of this logic; that teachers should be ignorant and left in the dark about the VAM, and this will make them feel “added value”? Are all Board members knowledgeable of the VAM, so that they can have meaningful discourses about it with their constituents?
If we “VAM unto others as we would have others VAM unto us”, then what kind of rubrics do we use with administration, district and state education leaders, and the legislators that make the mandates?
District administrators may feel this is unfair. Some might even admit to “uneasy feeling” of seeing their teachers get punished by the VAM (next year when all the finalities of SB 736 are implemented, teachers could see smaller pay for lower VAM), yet with no consequence to themselves. Though, if students do poorly because of poor teachers (though research says that that is only 14% true), then poor teachers reflect poor administration? Though administrators are never “vammed”?
Administrators, local and district, may complain that variables beyond your control should not be the reason for you lower paycheck under a Administrator-VAM statistic, but sorry no double standard. District leaders may grumble that FL DOE leaders, or FL legislators, made choices that negatively affect their performance at the district level. Well, so, should you not be held accountable anyway? Should there not be a VAM to unfairly punish you, as the one being used for teachers? FL DOE leaders may blame the federal government for poor funding, bad curriculum mandates, unsound/invalid pedagogic assessment models, and whine that getting smaller paychecks is unfair.
After all, should we not all just blame the President? No, of course not, individuals should and must be held accountable for producing excellent products; the “buck must stop somewhere”, agreed. Yet, why does it stop at the teachers? Why are we the only ones who will have a part of our paycheck tied to a VAM statistic? Why not leaders too? Why not parents too? Hey, why not penalize the future salaries of students who deliberately choose not to learn with a student-VAM?
But no, we, the teachers, will take all the blame for failing students (not even the parents get blamed) without any accountability (tied to salary) for our leaders? We are flattered that you leaders believe we have that kind of power in the classroom; that we can lead the horse to water and make it drink; that we can plan and cook the meal and make someone eat it too.
How we wish, as teachers, the assimilation of knowledge was so easy; that all our input equated to student output, but we all know this is false.
Does not Maslow’s hierarchy of needs teach us that no student will self-actualize and desire to learn for the delayed future reward of a good job, or the “love of learning”, whose underlying primary needs of love, support and home-life are not robustly provided. Students today, in general, come into the classroom with so much “baggage” (ex. uncertainty of family support after 2 divorces, abusive authorities, excessive premarital sex, an Internet that exposes them to less-than-desirable behaviors, having some of the adults they used to look up to disappoint them, being bombarded with inane, vain and useless internet entertainment, etc.).
So, teachers are expected to produce a superior product in spite of the defects in the raw materials that enter the classroom? As a chemistry teacher I believe I can help the student (ore) refine itself, and will do so with all my passion, but I cannot do alchemy; I cannot make efficacious teaching and learning happen when the ore is unwilling to be refined.
Do we even teach and test on the affective domain of hard work, perseverance, diligence, honesty, self-sacrifice, altruism, grace, mercy and self-control? Yet, are not there variables as important (if not more) to being successful in the real world, and are what most employers are looking for (not just cognitive potential)? Yet, our VAMs never consider these variables, and therefore are limited in their predictive power!
Did our test-metric-engineers forget about the maxim: not all that counts matters, not all that matters can be counted, not everything counted has value, and not everything of value may be counted?
After all, what do test scores prove, if not nothing more than having good test-taking skills (ex. cramming to fill short-term memory and organized mental-schema that help one store and access data). Research shows time and again that there is very little transfer of knowledge, or skills, across content areas in high school students because their underlying knowledge base and mental-schema are still in the developmental stages and have not had enough time and experience to make the deep and profound connections, that occur with more maturation and study.
We believe students should analyze and solve geometry proofs and logic-tests, hoping that it will transfer to their language arts classes, whereby they will be more able to deconstruct texts and analyze authors’ intent and purpose. Yet, research paints a picture that is more correlational than causational; students that can think, analyze and produce do so in most classes, regardless of the teaching strategy or assessments used. Which again, proves the goal of a good liberal-arts pedagogy of “teaching students how to think, not what to think”. “How to think curriculum” is more messy and sticky; it requires long-term studies and research, interdisciplinary explorations and teacher collaborations; it is not so neat and easy as teaching a “what to think” class with an EOC. Yet, how-to-think skills are used more in the real world, than the short-term (shallow measure) memory skills of EOC tests and the curriculums determined by them. After all, if a test goes into VAM, and VAM affects paycheck, then teacher will more likely “teach to the test”, and the overall quality of education will suffer…..duh!
In the end of it all, the Big-Picture, I always point out to my students the “Graph of School Predictions”, where the Predictors go on the X-axis (ex. GPA, AP classes taken, AP exams passed, FCAT scores, etc.) and Criterions go on the Y-axis (ex. future socio-economic class, contribution to the GDP, “being a prepared and responsible citizen”, “self-value/actualization/fulfillment”, etc.). Then, I point out that there is little, to none, long-term research and data to show that there is a positive correlation between predictors and criterion. We assume students with higher GPA will make a greater contribution to the nation’s GDP, but where is the convincing data. What about the outliers (if the real pattern is even known), like the Una-Bomber, who probably scored high in school, but was very-low in life?
In the Big-Picture, 90% of my chemistry students will forget 90% of the content on an EOC within 9 months, unless they restudy it because they take college chemistry. That can be said of most classes; unless the content is repeated and restudied (and then “permanently” assimilated), the learning was just temporary (and any short-term measure of it, ex. EOC: end-of-course exams) and has little to no effect and/or value in one’s adult life and career choice. So, why do we make such a big deal of EOC in high school, when they don’t even do this in college? If our tests have little power/significance in preparing, or predicting, success in the adult-life, then why are they such an important term in the VAM metric? Previous generations never suffered through all this testing, and teachers never scrutinized by VAMs, and yet the high school graduates of the 50s-80s were well prepared for college. So, what happened to the mantra: “if it ain’t broke, don’t fix it”?
So…..
Dear Superintendent, if we are going to get smaller paychecks because of an unjust VAM statistic, then why does it not work the other way? Why not divide up the money won for the Broad Award (or all other awards for education) and share the winnings with your teachers? Though, we might feel guilty because if we do get higher wages because of VAM incentives, then should we not share some of that with our students. For they, after all, were the ones performing on the tests. Just where does “the buck stop” or the reward end? Should some merit-pay go back to the families (their love, support, concern and accountability) that are the number one predictor of student success?
Mr. Governor, should not a proportion of your salary also be tied to student test scores? Is not the principle “a servant is not greater than their master” be applied to you? If we fail in the classroom, then your leadership must have something to do with it? Of course, the families never get any blame?
Mr. Scott if the FL GDP does not rise during your tenure, should not your salary be impacted? You may complain that federal policy has tied your hands. Well, do you now empathize with how teachers’ input is limited? Teachers strive for excellence, in spite of many of the socio-cultural variables that inhibit learning (ex. multiple divorces, inane and excessive entertainment medias, etc.). So, please treat us with knowledge and respect we deserve, and therefore drop VAM policies. If you disagree, then please have some statistician write a FL Governor VAM metric (and load it with variables you have no control over, ex. Senate bills passed) and then you might be more able to empathize with teachers.
Dear FL State Legislators, you approved VAM, so then apply it to yourselves. 50% of your pay should be based on the number of bills your author, and that PASS the legislature. We don’t care how hard you work, how many joules of energy you expend, or the product/profit you produce. No, we only care if your bills get passed (metaphor for test scores). Oh, but you will complain that the rate of bills passed is due to variables beyond your control. So, deal with it; suck it up and be treated in the same way you want to treat your teachers.
Florida public servants, ex. Firefighters and Police, should there be a VAM for you too. Should your evaluations, and part of your salary, be based on actual crimes caught, or those that were prevented? Is it the fires you put out, or your daily service to the community, that matters? If you do a good job and crime rates drop, and there are less crimes to catch, then should your VAM score get lower and paycheck too?
I think we all see the fallacy of “product-only” VAM measures, that take no account of the daily service and commitment of the worker (ignore all “process related” activities). If only test scores matter in my VAM, then why should I even care about collaborating with my peers, helping set up labs for the science department, leading local trainings and peer mentoring, writing curriculum and seeking best practices (if they have no influence on an EOC exam)? Why go the “extra mile” in my daily teaching, when all FL cares about is the “one-inch” of test data.
If one really considers the possible implications of a “dog eat dog” competitive environment of teaching to VAM tests, where teachers only care about their own students’ performances, and nothing else (since a significant part of their salary is at stake), education will become a bleak and barren wasteland of test-obsessed pedagogy, diminished in real and diverse learning and low in any true cognitive stimulation.
So, to all leaders who are unwilling to have a VAM equations applied to their own salary, STOP the hypocrisy and join us in a more just and equitable society. Be willing to walk in our shoes, before you arrogantly and pretentiously tell us how they should be worn.
Mr. Rick Lapworth
Science Teacher, Felix Varela High School
15255 SW 96 St, Miami, FL 33196
Ed.S., NBCT 1999-2019
305-752-7900, x 3259, rm 259
Fax: 305-386-8987
How much more evidence do we need to drop the VAM approach from all teacher evaluation systems? How many times will intelligent people need to explain that it simply doesn’t work? Thanks for another important post.
They know they are selling snake oil. This evidence is meaningless to the reform crowd. There was no attempt to cobble together a valid and reliable teacher evaluation model. It was all about strapping artificially inflated failure rates to teacher evaluations.
It was all about supporting the false claim that ALL of America’s schools are just failure factories filled with lazy, under-worked and over-paid babysitters. Peer review is a far superior evaluation model but without the test score link, it was never even considered by the Obama/Duncan regime.
“Teachers account for between 1% and 14% of the variation in test scores” DOES need to be posted and sent to news agencies, etc. However, I think it needs to be “translated” or framed in a more consumable way for the public, parents, students, teachers, politicians (anyone not a statistician) who doesn’t know what “variation” means in this context if we really want to get the message out.
I’m not one (a statistician) but I presume it means a teacher’s effect on a student’s test score from one year to the next is very minor. In other words, when a student scores higher (lower too?) it’s mostly due to other factors besides the teacher.
“I think it needs to be “translated” or framed in a more consumable way for the public, parents, students, teachers, politicians”
What also needs to be framed is the more important idea that the school experience, that teaching and learning is about much, much more than standardized test scores.
“VAM Mechanics”
“Simultaneously good and bad!”
What the VAM has said
Teachers are like Schroedinger’s Cat
Both alive and dead
Perhaps we are to believe that the situation with a teacher is like that of quantum mechanics where Schroedinger’s cat can be simultaneously dead and alive?
That VAM can rate a teacher both “good” and “bad” at the same time (see Gary Rubinstein’s post here) is worthy not just of harsh criticism but of ridicule.
It’s just plain goofy.
Here is one of the unintended consequences:
In my area teachers are substituting in affluent school districts for a year or more until there is an opening. They know if they teach in poor schools, the test scores will be low and they will be considered “bad” teachers. To be labeled a “good” teacher, they need to teach in a “good” (i.e. affluent) school.
I’m glad the ASA is independent enough to publish the truth.
I am a bad teacher in a bad school in a bad district. I have accepted my fate.
Same here. The Education Committee in Ohio insists public education is socialism (Brenner), teachers are paid enough and subpar (Lehner), and low test scores and rankings mean a teacher loses their license (Stebleton). Also teachers should be treated like “everyone else” (Becker). Plus the governor wants to “break the backs” of educators (Kasich). These are just the public statements. One can wonder what our anti-education state leaders cook up in private.
Linda, I agree. The VAM system penalizes educators working with the most challenging students. The other unintended consequence is that teachers are forced into managing their classroom like a portfolio of students. The question is not “how can I help”, but now “who can I help”? In an affront to everything teaching is meant to be, the Reformers force teachers to focus only on those students who can provide the best “return” on “investment” – the best test scores from limited instruction time. Otherwise, the PARCC tests eliminate the teacher and even destroy a career. This perversion of education by the Reform movement is the cruelest aspect of VAM impacting students, parents, and teachers. What do these free market Reformers call it? “Creative Destruction”? Wonder if THEY are being destroyed or held accountable when VAM fails?
How about the American Statistical Association being an expert witness in a class action lawsuit on behalf of every teacher let go on the grounds of VAM??? And that is only the beginning.
Then some REAL education experts could be expert witnesses on behalf of public students across America whose right to a free and quality education has been violated by “testing” over “learning”…
The real insult of VAM, though, really (if you think about it) is to administrators. VAM says “we don’t trust your judgement, administrators.”
Insult?
The way it should be.
If we’re going to clean house in American public education–fine, let’s start all the way at the top.
They abandoned teaching kids and claim, “I can do more for kids in admin.”
Nobody trusts their judgment. They’re running for cover and making twice as much as the soldiers working with the children.
YEP!
In addition to posting the legitimate impact a teacher has on students’ scores, I would suggest that more teachers sue if test scores account for more than 14% of an evaluation, especially if it results in loss of pay. There is a lawsuit pending in Long Island on this issue, Other teachers and perhaps unions with class actions suits should file as well to challenge those that use voodoo science to evaluate staff. I don’t believe resignation is the way to go. If you walk into the showers, they will be more than happy to turn on the gas. Sometimes you have to fight fire with fire.
Another Tweet for this post that anyone who has a Twitter page may copy and paste
Teachers account for between 1 & 14% of variation in test scores
Ranking teachers by VAM scores can reduce quality
http://wp.me/p2odLa-8I2
I just wrote this when I shared the link: figured I’d post it here too… Currently all of our evaluations in the student growth area are required to be 40% of the determination of our ‘end of the year’ annual teacher evaluations… I believe we have new labels or maybe they are still ‘highly effective, effective, minimally effective, and/or ineffective’. When it comes to ‘staffing- who gets laid off, who stays’ (remember we are a right to work state so seniority is low on the totem pole for staffing’). These evaluations are based on two classroom visits and your evaluator can and is required to literally write down the good and the bad. If little Johnny is off task- you aren’t engaging him. The lessons you teach all year are narrowed down to two random classroom observations… Two years ago I was evaluated on a class while I was at home with shingles (I don’t know how either- maybe it twas’ magic) and last year I got ‘evaluated’ when I had pink eye and didn’t know it… I’ve never had a ‘bad’ evaluation but believe that the system is flawed for all of us who work in education. It’s not fair for admin or teachers and puts a lot of stress on teachers newer and more experienced. So anyways when it comes to ‘staffing’ the first thing considered is your end of the year annual evaluation label which comes down to a popularity contest and just doing as your told whilst biting your tongue (even if that means watching something implemented fail and be redesigned in the way you wanted to suggest but had to bite your tongue- just take that as an example). The second thing that is considered is your attendance… Say you take off 6 days in the school year for whatever reason and your colleague has the same evaluation label as you but they took off 5 days and not six… guess who gets laid off? the person who took less days off. In addition if 40% of your evaluation is required to be based on test scores and say you teach at a school that has less growth for whatever reason (socioeconomics, transiency, geographic location in your school district) guess who gets laid off, you do. I do not believe that this structure will develop much further as it seems to be very expensive and very hard to analyze to improve, teaching is an active social science that can’t be put into such a box. I do like how the new NWEA testing and other new expensive computer program assessments show individualized student growth over the years, however, tying data to evals and the entire evaluation system is in my humble opinion, flawed. Congrats to me for procrastinating by writing such a long post! I shall work on some jade leaf propagation now. Happy SATURDAY!
I just wrote this when I shared the link: figured I’d post it here too… Currently all of our evaluations in the student growth area are required to be 40% of the determination of our ‘end of the year’ annual teacher evaluations… I believe we have new labels or maybe they are still ‘highly effective, effective, minimally effective, and/or ineffective’. When it comes to ‘staffing- who gets laid off, who stays’ (remember we are a right to work state so seniority is low on the totem pole for staffing’). These evaluations are based on two classroom visits and your evaluator can and is required to literally write down the good and the bad. If little Johnny is off task- you aren’t engaging him. The lessons you teach all year are narrowed down to two random classroom observations… Two years ago I was evaluated on a class while I was at home with shingles (I don’t know how either- maybe it twas’ magic) and last year I got ‘evaluated’ when I had pink eye and didn’t know it… I’ve never had a ‘bad’ evaluation but believe that the system is flawed for all of us who work in education. It’s not fair for admin or teachers and puts a lot of stress on teachers newer and more experienced. So anyways when it comes to ‘staffing’ the first thing considered is your end of the year annual evaluation label which comes down to a popularity contest and just doing as your told whilst biting your tongue (even if that means watching something implemented fail and be redesigned in the way you wanted to suggest but had to bite your tongue- just take that as an example). The second thing that is considered is your attendance… Say you take off 6 days in the school year for whatever reason and your colleague has the same evaluation label as you but they took off 5 days and not six… guess who gets laid off? the person who took less days off. In addition if 40% of your evaluation is required to be based on test scores and say you teach at a school that has less growth for whatever reason (socioeconomics, transiency, geographic location in your school district) guess who gets laid off, you do. I do not believe that this structure will develop much further as it seems to be very expensive and very hard to analyze to improve, teaching is an active social science that can’t be put into such a box. I do like how the new NWEA testing and other new expensive computer program assessments show individualized student growth over the years, however, tying data to evals and the entire evaluation system is in my humble opinion, flawed. Congrats to me for procrastinating by writing such a long post! I shall work on some jade leaf propagation now. Happy SATURDAY!
I wrote this when I shared the link to the article and figured I’d share it here… : Currently all of our evaluations in the student growth area are required to be 40% of the determination of our ‘end of the year’ annual teacher evaluations… I believe we have new labels or maybe they are still ‘highly effective, effective, minimally effective, and/or ineffective’. When it comes to ‘staffing- who gets laid off, who stays’ (remember we are a right to work state so seniority is low on the totem pole for staffing’). These evaluations are based on two classroom visits and your evaluator can and is required to literally write down the good and the bad. If little Johnny is off task- you aren’t engaging him. The lessons you teach all year are narrowed down to two random classroom observations… Two years ago I was evaluated on a class while I was at home with shingles (I don’t know how either- maybe it twas’ magic) and last year I got ‘evaluated’ when I had pink eye and didn’t know it… I’ve never had a ‘bad’ evaluation but believe that the system is flawed for all of us who work in education. It’s not fair for admin or teachers and puts a lot of stress on teachers newer and more experienced. So anyways when it comes to ‘staffing’ the first thing considered is your end of the year annual evaluation label which comes down to a popularity contest and just doing as your told whilst biting your tongue (even if that means watching something implemented fail and be redesigned in the way you wanted to suggest but had to bite your tongue- just take that as an example). The second thing that is considered is your attendance… Say you take off 6 days in the school year for whatever reason and your colleague has the same evaluation label as you but they took off 5 days and not six… guess who gets laid off? the person who took less days off. In addition if 40% of your evaluation is required to be based on test scores and say you teach at a school that has less growth for whatever reason (socioeconomics, transiency, geographic location in your school district) guess who gets laid off, you do. I do not believe that this structure will develop much further as it seems to be very expensive and very hard to analyze to improve, teaching is an active social science that can’t be put into such a box. I do like how the new NWEA testing and other new expensive computer program assessments show individualized student growth over the years, however, tying data to evals and the entire evaluation system is in my humble opinion, flawed. Congrats to me for procrastinating by writing such a long post! I shall work on some jade leaf propagation now. Happy SATURDAY!
Man you’re an expert at procrastinating-writing the exact same thing three times and posting it three times!!!!
Hi Duane, Thanks for judging my writing. In case you didn’t notice the first sentence; I copy/pasted what I wrote for another social media site. Are you a teacher? Do you understand that it is difficult for non-teachers to understand the scrutiny we face. Thanks again for responding to my comment! Have a great day! You’ve inspired me to empower people I come into contact today rather than to belittle them. Bullies need the most love.
sruttan2014,
I see that once again my attempt at humor has failed miserably. Please reread with a grain (or perhaps a box of) salt so my humour might come through. It was not meant as a jab at ya!!
Love & Peace
Duane
Good evening Duane, Sincere apologizes for being absent-minded and miss reading your comment to my post. I didn’t know that it was posted three times and immediately went on the defense for believing you were under the impression that my post was repetitive, and redundant. Only after I re-read my post and realized that I mentioned procrastination at the very end did I realize that you read my long post and were making a joke of me posting it three times. I was repetitive in my post and didn’t proof read or edit it (which a constant work in progress on my part, impulsive- hehe). I viewed your profile and read the beginning of the work you did regarding admin being more business minded and am intrigued by your intuition and receptiveness. Best wishes, Shauna PS- I didn’t proof read this either, maybe it’s the art teacher in me, spontaneous! ☺
No problem, Shauna. If we were all perfect life would probably be boring.
By the way, what is NWEA stand for? I’m AI (self diagnosed) and need all the help I can get in figuring out educational acromyms.
And if you don’t mind what state are you in that requires that 40% be a part of the evaluation?
Gracias,
OFST
(old fart Spanish teacher)
Hi again Duane! Silly me! Did you know that I haven’t ever posted a comment before and I didn’t know my WordPress password? I Had no idea it posted three times.. THANKS AGAIN!….
Please come back and post more and maybe you’ll find that sometimes my humour works and other times it doesn’t. I keep on trying though. And yes, I am a teacher and sometimes my attempts at humour work in class and other times they flop.
“VAM: The Scarlet Letter”
Reblogged this on Network Schools – Wayne Gersen and commented:
If VAM is statistically invalid for classroom teachers who provide direct instruction in classes that instruct students in tested material, how could the use of VAM be valid in ANY way for SpED, Music, Art, PE, MS Science, MS Social Studies, the majority of HS content areas, etc. etc.? By my back of the envelope calculation a majority of teachers could not be measured by VAM even it it WAS a statistically valid calculation. My fear: Lamar Alexander and other reformers will solve this problem by giving more tests to more children in more subjects.
Wait a minute, do you think legislators who follow an “agenda” really care about statistics? ARe you under the impression that many who are elected and care more about a political agenda are really going to listen to statistics?
We’ve watched school board members for YEARS ignore the pleas from parents on the damage by fuzzy math.
We’ve seen legislators ignore the warnings of developmentally inappropriate curriculum based on Common Core standards.
AGENDAS drive these people. They believe in a UTOPIA where facts and statistics do not matter.
We see the same UTOPIA among the teachers unions that push extreme political agendas.
Let’s face it…political HACKS do not care about teachers.
Can you tell me exactly what utopian ideals teachers’ unions are pushing? My union is pushing for schools kids deserve. We haven’t had a raise in 8 years, worked without a contract for two, had our class size increased, had Common Core dumped on us with very little training, saw our librarian, nurse, aides, office clerks, and janitors taken away, protested while repairs went undone so I-pads could be purchased at top dollar, currently being used at about 5% of the schools. The reformers championed a guy who left us in a mess. I don’t think this is a case for a pox on both their houses. What have the unions in your area done that is so outrageous that you think they are equally culpable?
Many believe we need to fix the system…. Unlike every other buisness in the world when they have supplies that don’t meet their standards or is not a quality material, they go someplace else. Is the teaching profession given that ability, no. But yet we are expected to produce the same thing from 32 different students. It’s time we look at reforming the suppliers of the classroom.
True. Businesses in private sector choose their customers.
The analogy I was using this past week involves home repairs. Imagine your house needs a new roof or a new bathroom or even a new foundation. But then some big multi-national construction corporation shows up with a scheme of its own -not only to fix what your house might need but to renovate the entire place, top to bottom. Plus, they’re here to “fix” every other home in town, too -with the EXACT SAME “PLAN”.
So, you’re trying to have a rational discussion with the foreman of this alleged construction crew then you hear a chainsaw kick on down the hallway. You run down there just as some guy starts ripping apart your kid’s bedroom furniture, splinters flying. You can barely hear what the smiling worker is saying over the din but you get the idea when he hands you your own chainsaw and motions for YOU to join in, too. What the hell is the plan, you scream. “We…don’t….have….one”, he yells back, “We’re building the plane as we’re flying it”
WHAT….?
They are destroying your entire house. But, of course, it doesn’t really matter to them. If they burn down the town from one end to another, they’d only be happier. They don’t want you, they don’t want your house.
Their plan to renovate the world reminds me of the wacky “modern” architecture that leveled entire neighborhoods in cities 50 years ago and left us with mammoth housing projects….the urban “renewal” that ended up being worse than the tragic problems it sought to cure. It was all so much about science and “progress” back then, too, you know. I reread Tom Wolfe’s critique of modern architecture (“From Bauhaus to Our House”) a few years ago, when the Race to the Top crap first really hit the fan. How could such madness grip an entire culture and come to shape our world? I just had to shake my head…..yup, here we go again.
The statistical argument about VAM is great….it’s powerful……I’ve used it myself since reading about it on this blog. We should shout it from the rooftops. But I don’t think some of the people who are attacking public schools and destroying what goes on in our classrooms really care.
It’s like a good carpenter thinking about how to use a specific tool…..meanwhile a giant, smoke-belching excavator is sidling up on its greasy tracks, crushing whatever is in its way and swinging the good ole wrecking ball……
“Education Reformers”
They’re not about “fixing” schools
But wrecking them instead
Replacing them with tools
To make them loads of bread
Thanks, SomeDAM Poet. Love your stuff.
I decided to do a career change and become a teacher; to have an impact on students’ futures. I tell them my job as their teacher is two-fold. First And foremost is providing them some math tools to be used in their future, whether it be in a 4 year college, 2 year tech college, or a career without further schooling. My secondary job is to prepare them for life to be productive, contributing, and successful citizens. This means I teach them explicitly and through examples how to handle confrontations, disagreements, proper manners, behavior, & above all, that ties it together, respect. Teaching freshman, Algebra 1, numerous times throughout the year the students will ask, “when will I ever use this?” I have 2 replies at this point as being in my second year: The 1st is provided courtesy of the government, “Because politicians, not educators, say you need to learn this.” And my other reply is “How Education.” This typically provides a platform for discussion on politics’ involvement in their education; and, how, typically, educators are not involved in such decisions.
I don’t need a test or politician to tell me I have succeeded in my goals of preparing our youth for their futures since at least half of their schooling doesn’t involve the subject I teach, math. My rewards and satisfaction come from students telling me how they remembered something I said over the weekend when they were faced with a potentially life changing situation and it influenced the way they decided to handle that situation. All yeah, my initial reason for getting into teaching is another reward; seeing a student’s smile when they have that “Ah ha! Moment of understanding” is another way I know I’m accomplishing my goals of preparing our youth for their futures.
Being a 2nd year teacher, with 20 years of real world application experience, that is my perspective.
Poet, You really need to publish all of your poems in a book for teachers. I would buy a copy definitely! Your poems are awesome, and I always look for them on the blogs! Thank you so much for your humor! We all need that humor so much! It feels good to laugh when we are all under so much stress!
It is based on the Confucian ideal that there are no bad pupils, only bad tutors. We need to test students somehow to make sure they meet basic writing and math scores, what is the big deal? Teachers need to be held accountable somehow, how else do you evaluate them? If one or two kids fail, that’s one thing; but testing is otherwise a good way to weed the bad from the good. Teachers have it too easy to get and keep a job, and few ways to weed the bad from the good!
Do you blame doctors for the flu?
I’ll agree with this much: VAM is based on Confusion. 🙂
It’s “VAMdom Crazy Mad”
VAM weeds bad from good
And VAM “weeds” good from bad
Quite oft misunderstood
And random crazy mad
As the American Statistical Association point out in their VAM position paper (here because they have large standard errors, VAM scores are “unstable”
What this means is that a teacher’s VAM score can jump around a lot (from one value added model to another and from one year to the next), due to factors out of the teacher’s control.
Value added models might aptly be called “Jumping VAMs” for this reason and are simply unreliable as a means of ranking individual teachers. One year a teacher can be ranked “cream of the crop’ and the very next, “bottom of the barrel”.
As Gary Rubinstein shows here for example, in many cases, VAMs give essentially random results and can even rate the same teacher both “good” and “bad” simultaneously, which is just pure nonsense.
Might just as well throw dice to decide which teachers should be rated “good” and which “bad” (and canned). Of course, if a teacher is fired after being ranked “bottom o f the barrel”, they don’t get another throw of the dice.
If anyone ever needed evidence that a poor/invalid “measure” can actually be worse than none at all, VAM is it.
VAMpire is there to test students how healthy their blood works for bogus high-stakes standardized testing. It doesn’t measure anything about reading/writing. It gives unaccountable accountability seekers
license to kick teachers out of school for low scores of students they don’t teach or have in the classroom?? What’s the solution? Well, why don’t you bite garlics and breathe it out to unaccountable education authorities who are bitten by VAMpire???
I had a horrible class last year because of all the behavior problems.. now my vam score states I’m not effective..the school I work at is one of the worst schools here, now I’m stuck there because no-one is going to hire me of I’m not effective! Lol my life is over
Your life isn’t over. Some years ago there was a study that discovered that seasoned teachers who left the profession for the private sector, on average, ended up earning more money within a few years than they would have earned if they had stayed in teaching.
The job of being a teacher requires developing many skills that most workers in the private sector will never acquire. For instance, there was a teacher at the high school where I worked who had an undercover security job at Disneyland in Anaheim every summer because Disneyland preferred to hire teachers during their peak season when teachers were out of work due to the fact that most teachers could spot trouble before it happened and had advanced people skills to handle the challenge and/or defuse it quickly.
I’ve started tutoring privately. It’s been less than four months and I’m already earning basically my teaching take home pay, at just over half the teaching hours (I still plan, but no grading). I’ll be at the point of turning people away soon; my “evaluation” is word-of-mouth recommendations among clients based on results, which has built up business quickly. So, basically, you probably ARE exceptionally good at your job. You may need to figure out another environment outside of the public school madness to be able to actually effectively do your job, though, since the increasing lack of autonomy there has made it nearly impossible.
How did you develop a customer base?