Education Week reported that a decade of “reforms” focused on tougher teacher evaluations produced no improvement in student test scores.
More than a decade ago, policymakers made a multi-billion-dollar bet that strengthening teacher evaluation would lead to better teaching, which in turn would boost student achievement. But new research shows that, overall, those efforts failed: Nationally, teacher evaluation reforms over the past decade had no impact on student test scores or educational attainment.
The research is the latest indictment of a massive push between 2009 and 2017, spurred by federal incentives, philanthropic investments, and a nationwide drive for accountability in K-12 education, to implement high-stakes teacher evaluation systems in nearly every state.
Prior to the reforms, nearly all teachers received satisfactory ratings in their evaluations. So policymakers from both political parties introduced more-robust classroom observations and student-growth measures—including standardized test scores—into teachers’ ratings, and then linked the performance ratings to personnel decisions and compensation.
“There was a tremendous amount of time and billions of dollars invested in putting these systems into place, and they didn’t have the positive effects reformers were hoping for,” said Joshua Bleiberg, an author of the study and a postdoctoral research associate at the Annenberg Institute for School Reform at Brown University. “There’s not a null effect in every place where teacher evaluation [reform] happened. … [But] on average, [the effect on student achievement] is pretty close to zero.”
The evaluation reforms were largely unpopular among teachers and their unions, who argued that incorporating certain metrics, like student test scores, was unfair and would drive good educators out of the profession. Yet proponents—including the Obama administration—argued that tougher evaluations could identify, and potentially weed out, the weakest teachers while elevating the strongest ones…
A team of researchers from several universities analyzed the data, starting when states adopted the new teacher evaluations incorporating student test scores. They looked not only at changes in scores but high school graduation rates and college enrollment rates.
Tougher teacher-evaluation systems can work, Petrilli said—but there was no political will to act on the results at the time of the reforms. Teachers’ unions resisted firing teachers who received poor results, and districts were unwilling or unable to pay great teachers more, he said.
At a time of acute teacher shortages, what school district is eager to fire teachers based on their students’ test scores?
The failed reforms were in large part a response to the demands of the Obama administration’s Race to the Top, which required states to adopt test-based evaluation to be eligible for a share of $4.35 billion in federal money. Secretary of Education Arne Duncan praised such teacher evaluations loudly and frequently.
As I wrote in my 2020 book SLAYING GOLIATH, test-based teacher evaluation was never tried before it was imposed on almost every state in the nation. It had no evidence to support its use. Many scholars and professional groups warned against it, but Duncan plunged forward, belittling anyone who dared to disparage his Big Reform.
Obama and Duncan found support in a 2011 study led by Harvard economist Raj Chetty, but his glowing predictions about the benefits of test-based evaluation didn’t pan out. His paper on value-added teacher assessment won him a front page story in the New York Times, a story on the PBS Newshour, and a laudatory mention in President Obama’s 2012 State of the Union address. Chetty et al concluded that better teachers caused students to get higher test scores, to graduate more frequently, to earn more income over their lifetimes, and—for girls, to be less likely to have out-of-wedlock births. As one of the authors told the New York Times, the message of our study is that bad teachers should be fired sooner rather than later.
But despite the cheerleading of Arne Duncan and the seemingly definitive conclusions of Chetty, Friedman, and Rockoff, value-added teacher evaluations failed.
How many good and great teachers left their profession because of this ill-fated “reform”?
Well, well, well….So the “War on Teachers” turned out to be nothing but – a war on teachers.
The anxiety, depression and suffering teachers have been put through should make every politician and educrat feel shame. The profession has been demoralized for too long now. I pray it can one day become an honored, desired profession again. Many of my younger colleagues are planning to leave as soon as possible. Some have already left. Shame.
In my many years as a student and intructor and parent of 5 kids and grandparent of 6, involved closely in their educations (in the classroom and lately helping grandkids with zoom learning), there are in fact some teachers that are not good. Very few. I wonder what can be done to increase the quality of teachers and teaching. I have always thought that putting teachers in charge of schools and eliminating administrators who know little about teaching would help. Allowing teachers to teach and control curriculum would help. Getting rid of “professional development” that has nothing to do with teaching would help. Encouraging or allowing teachers to spend more time collaborating would help. Putting more money to pay teachers and improve buildings and classrooms in low income minority districts would help. Just some of my thoughts.
The concepts behind Japanese Lesson Study would go a long ways to provide collaboration time among teachers. It also allows teachers to see colleagues teach, to improve the lessons and to discuss ways to make them better so students learn. Of course, this takes time, money and more flexible scheduling than most school systems and administrators would ever allow.
Having participated in an inservice training program that encouraged collaboration with and observation of peers, I can say it was a most enlightening experience. As teachers, we rarely get out of our classrooms to see each other at work. I was greatly impressed by most of my peers, finding many of them much better than I supposed. The point here is that to understand the abilities of teachers, one must see them at work, not rely upon test scores, which may tell us what a student has or hasn’t learned, but not why they learned or did not learn it.
Kenneth, you are so fortunate to have been able to do this. As a dept chair, I wanted to be able to implement some parts of lesson study b,it ran onto inflexible scheduling and insufficient time.
cx: but
The concepts behind Japanese Lesson Study would go a long ways . . . . Of course, this takes time, money and more flexible scheduling than most school systems and administrators would ever allow.
Yes, yes, yes, yes, yes!!!!!!!
Countries that are credited with good results, such as Singapore and Finland, focus on teacher preparation and meaningful time to plan and collaborate. That would be a good start here. Too often win the United States, teaching is not regarded as a profession. If we could change that culture quality would improve.
My child had a “not good” teacher for 1st grade (a very seasoned educator) who turned out to be the best teacher for her and the one she still mentions often. I think many teachers need time to learn how to “herd cats” while teaching what needs to be taught.
The “powers that be” decided that they could increase the quality of teachers/teaching by adopting nationwide standards (Common Core). Any person could deliver CC instruction. The “powers that be” had to test the kids to death to see if the teachers were/are doing a bang up job of teaching to “The Core”. The bad curriculum and the stupid testing go hand in hand in making schools/teachers “look bad”.
Lisa
You highlight a critical aspect that the VAMers miss. And they miss it quite on purpose because it is very inconvenient and does not fit in with their overly simplistic view.
Good teaching is not primarily about test scores.
Lots of the things that good teachers do for students can’t be readily gauged.
Certainly not by some unidimensional standardized test that is designed and used precisely because it is so hard to gauge good teaching and so easy to come up with a (fake) “measurement” system that claims to do so.
Economists love standardized tests because they produce reams of scores, data that can be manipulated to the nth degree. It doesn’t matter to the economists that the output from all their manipulation is rubbish because it’s all about the mathturbation — and the “prizes” you can get for it.
Ofc, it matters not a whit that the purported “data” is worthless because it is based on invalid tests that do not measure what they purport to measure. GIGO.
That’s Duane Swacker’s cue:
Strictly speaking , the tests don’t “measure” anything.
Real measurement requires a precise definition of what one is measuring that is completely independent of (ie, does not depend on )the specific instrument (in this case standardize test) used to perform the measurement. Real measurements can be performed with different instruments and yield the same results (within the uncertainty of the instruments).
Real measurements also always include a unit of measurement that, again is independent of the specific instrument.
Supposed “measurements” performed with standardized tests don’t meet any of the normal criteria for proper scientific measurements.
As Duane has pointed out many times, this is not merely some meaningless issue because the term measurement means something very specific in a scientific sense and to misuse it is to effectively timely try to co-opt the terminology and make a pseudoscience seem to be scientific.
The whole thing is circular:
What does a particular standardized test measure?
Educational achievement.
What is educational achievement?
What(ever) the test measures.
If that sounds circular and silly, that is because it is.
And sadly, it is also an accurate representation of the situation.
Well stated, SDP! As mentioned by Robert: GIGO which is the nice way of stating Crap In = Crap Out. Or invalid pseudo-measurement results in invalid assessment as we’ve seen from the standards and testing malpractice regime.
It amazes me that someone like Jlsteach can’t read and understand that simple concept. (Yeah, trying to suck you into the conversation Jlsteach, couldn’t resist.)
And most definitely it is a misappropriation of terminology to call a standardized test a measurement. . . of any kind.
Not coincidentally, the arguments regarding VAM also exhibit the same circularity
What does VAM measure?
Student growth
What is student growth?
What VAM measures
Who are the good teachers?
Those who get high VAM scores
Who gets high VAM scores?
Those who are good teachers
Move over Ptolemy!
content://com.android.chrome.FileProvider/images/screenshot/16384524743761338390658.jpg
Make way for Chetty et al.
SDP,
Isn’t Chetty’s celebrated work on VAM simply an illustration of correlation, not causation? Students who get high test scores are likely to graduate from high school, enroll in college, and earn more than students who got low scores.
Move over Ptolemy!
https://www.britannica.com/science/Ptolemaic-system
Make way for Chetty et al
The “correlation isn’t causation” issue is just the tip of the Chettyberg.
Poor Chetty thought he was going to get a “Nobel” for his VAM paper but instead all he got was egg on his face.
I served as a mentor to new teachers for several years. It is a non-threatening, collaborative way to help teachers build capacity. Unlike administrators mentors are not there to evaluate. They are there to guide, suggest, model and encourage.
The evaluation battle may have been won through research, but the floggings continue. Teachers are still disrespected, underpaid and overworked. There is a new boogeyman in the room based on disinformation. It is the anti-CRT and liberal propaganda belief that the right insists is a major issue in public education despite the fact it has never really been an issue….until the right decided to invent it for political gain. Public education continues to be a national punching bag. All the stories about so-called education problems, real or imagined, make the evening news. I doubt the debunking of test based teacher evaluation will make it on the mainstream news. Now that the test based evaluation issue has been resolved, when is someone going to take a legitimate look at the impact of standards in education?
After years of Deform, it’s a wonder that ANYONE wants to teach in K-12 still. And it’s no wonder that kids in college are staying away from teacher development programs. Who in their right mind would want a job with those kinds of work conditions?
cx: his or her right mind
Shouldn’t discriminate against the bots
What in it’s right mind?
Make that its
Damned autocorrect bot.
Maybe that should be ” what in its right neural net?” since we don’t yet know whether bots can have minds.
This is what has had the greatest impact on teacher attrition. After the Wall Street collapse in 2009 our superintendent proclaimed that this was the opportunity to remove teachers through performance measures. The state of North Carolina had already cut the education budget and Charlotte was required to release 500 teachers. Test scores were part of the algorithm, but so were other subjective measures like observations. The worst part of my tenure as a principal in NC and Alabama was navigating teacher evaluations that were not only unfair, but often unethical. In my time as a teacher and administrator I have come to the conclusion that our investment should be in teacher preparation and support (including dramatically higher pay). In this capitalist bastion we call America, money matters. We need to show teachers that we will provide all of the tools needed to help them succeed. To keep incompetence to a minimum our preparation programs should include a demanding liberal arts background with significant internships. There are many I have met in my life who would have loved to teach. However, they are not blind. The lack of regard shown by our polity and the public education establishment is there for all to see. No Child Left Behind, the Great Recession, Race to the Top, and now the pandemic represent a perfect storm that threatens to demolish our teaching corp.
Paul: I wonder what practices you were required to take part in that were unethical?
Not Paul, but will give it a try. The #1 is the topic of this post. . . implementing the standards and testing malpractice regime. I’ll let others add the many more that we as teachers see on a daily basis.
We had a district “inquisition” every year that used a rubric based on observations, test scores, attendance, discipline referrals (the more referrals, the lowest the score) etc. to determine whether a non-tenured teacher (less than three years experience) could have a contract renewed. I participated in one of these events and was not invited again, probably because I sat there stunned at the unprofessional process. Teachers who had maternity leave were docked for attendance just like those who were absent for no reason because it was determined not to be fair to be treated otherwise. I lost a very promising young teacher due to a one time 10 minute walk through conducted by a district rep. The environment in the schools and district was terribly toxic because young teachers lived in fear of losing their jobs.
I agree wholeheartedly. I attended a liberal arts university, Sewanee, where we were not allowed to focus on vocation as a major. I simply had to take courses that would line up with certification in Tennessee along with my Fine Arts degree. Two years after graduating I started my masters in art education where we took a series of MFA level studio courses. I was astonished how unprepared many of the Art Ed folks were in regard to the technical and creative expertise required for these classes. I firmly believe that content should come first if we are to offer students a meaningful understanding of subject matter. I then believe that we should require graduate study that is heavily dependent on internships that offer a stipend was incentive. As a principal, I came to the conclusion that not only do teachers need to have profound people skills, but that a toughness is required to be prepared for the slings and arrows from the school house and community. I have also come to the opinion that perhaps teachers should have full classroom duties at 25 rather than 22, maturity matters. I loved the classroom and hated to leave it. I taught in urban schools and learned that my experiences with a variety of people in my then short life prepared me to interact with underprivileged kids. My oldest daughter just had a similar experience in an urban school, but left to teach at an independent school after five years in a public system that was unsupportive. I believe the only way the public schools recover is if the focus is on teacher preparation, support, and collaboration. Treat them as the experts and get out of the way. Priorities at the district and state level have let us down too often.
Amen, Paul! Yes. Yes. Yes.
Far and away the best manager I ever had in publishing was a guy with the altogether appropriate name of Bill Grace. He once told an all-hands meeting of our company, “I am going to share with you the secret to my success: I hire people who are a lot smarter than I am and get the ____ out of their way.”
Well sai, Paul.
said
Has anyone pointed out that it is irrational to expect that this year’s, say, fifth-graders would score higher on a test than last year’s fifth graders? Unless there has been some sort of transformation in the students, in the curriculum or teaching methods, etc., the only rational expectation is that this year’s crop will do as well as last year’s. And what do we see–“flat test scores.” Ta da!
Ways to improve the “crop” of students is to make sure they aren’t hungry, find more engaging ways to teach them, create better instructional materials, invent better teaching methods, etc. So, what is being done by these people who want higher test scores on these fronts? Nothing. It is as if they are trying to improve the intellectual level of television programming by adjusting the brightness settings on their sets. These people haven’t a clue but they do have another agenda, and it isn’t pretty.
If a poorly performing NFL football team stayed pat with the same players, coach, etc. during the offseason, what would their fans expect of their future performance? Yep, dismal results again. If football fans can figure this out, what’s wrong with these so-called education experts? They should be clamoring for better everything, including funding (poor team fans always want a new owner with deep pockets” who will bring in better coaches, players, etc.).
What you suggest is precisely what the American Statistical Association said in their 2014 statement on VAM
the majority of opportunities for quality improvement are found in the system-level conditions. ”
But of course, none of the people in charge paid any attention.
They were too stupid and/or too dishonest to “get” what the ASA and others were saying.
Has anyone pointed out that it is irrational to expect that this year’s, say, fifth-graders would score higher on a test than last year’s fifth graders?
This is supposed to happen via magic, Steve.
These people haven’t a clue
This is the breathtaking thing. We are decades into the Deformer Occupation of U.S. schooling, and the incompetence of these people and the stupidity of their magic formulas is breathtakingly obvious. And, THERE HAVE BEEN NO POSITIVE OUTCOMES. Zip. No increase in average test scores. No closing of the achievement gaps. And yet we go on, devolving our curricula to make it ever more test preppy, staying the insane course.
“Has anyone pointed out that it is irrational to expect that this year’s, say, fifth-graders would score higher on a test than last year’s fifth graders?”
That question wins the gentleman a large Kewpie Doll!
Just one of the many unethical inanities/insanities that are a part of the standards and testing malpractice regime. There are many others.
“And what do we see–“flat test scores.””
Because that is exactly the way the test is designed. The have reliability the test must have the same or very similar range of scores as the previous one. A test may be reliable. . . reliably invalid. Invalidity trumps reliability for without validity every aspect of the standardized test malpractice regime process is an exercise in mental masturbation.
To have reliability, not “the have”
The people like Eric Hanushek who pushed VAM had a history of being reliably stupid so they should have been completely ignored.
Now, if they had been unreliably smart, there might have been some reason for listening because there would at least have been a chance that what they were proposing might actually work.
But not when they were just reliably stupid. No reason at all for listening in that case.
Erik Hanushek recently won a $4 million prize for his research in education. If you recall, he maintained that if children had five great teachers in a row, the achievement among groups would close. I don’t think there are any living examples of this theoretical scenario. Who is a great teacher? The ones whose students get high test scores. Hanushek also has argued that the way to get better teachers is to fire the lowest-performing teachers every year, who will be replaced by average teachers. Again, scribbled on a napkin, but no evidence. Churn, churn, churn. Who will want to teach the kids who get the lowest test scores? A sure way to get fired. Apply for jobs only in wealthy suburbs.
Like I said, Hanushek is reliably stupid.
He’s what is known as a statustician: someone who is primarily concerned about his status.
Chetty (aka VAManujan) is another prime example of a statustician.
VAManujan (The Man who knew InVAMity)
The man who knew InVAMity —
Equations from the stars —
Who pushed the VAM insanity
From Earth to distant Mars
The results of this debacle are now seen in the dwindling enrollment in education schools. The number of math ed people who graduated from our local state university, a typical teachers college, last year was exactly one (a former student of ours). This is an unsustainable model, and no one at a state level is doing anything to remedy it.
The reason teacher evaluation failed here is that it was based on testing. If a teacher got poor test scores, that meant he got poor evaluations for the most part. Evaluators who gave ratings too divergant from the teacher’s test scores were hassled by the state. Headlines read: The test this year will only count 10% of a teacher evaluation. This was pure manure. The test drove everything. Garbage in garbage out.
I am cynical enough to believe that the political powers that be wanted exactly that: a crisis they could ignore. The house is burning and they will not even call the fire department.
What a surprise (not).
The American Statistical Association noted in their 2014 statement on VAM that
“Most VAM studies find that teachers account for about 1% to 14% of the variability in test scores, and that the majority of opportunities for quality improvement are found in the system-level conditions. Ranking teachers by their VAM scores can have unintended consequences”
Let that sink in.
A correlation between two variables that accounts for between 1% and 14% of the variability.
Whoopdy Doo!
For any real science, such incredibly weak correlation (and it’s only a correlation, not necessarily a cause effect relationship) would be considered essentially meaningless, but to some economists, it means “Hello, Stockholm. we’ll get a Nobel* Prize for sure” (* albeit a fake one, cuz the economics Prize is not a real Nobel: https://fivethirtyeight.com/features/the-economics-nobel-isnt-really-a-nobel/
The monkeys who pushed this stuff don’t understand basic statistics, so it’s no surprise that they would have pushed it.
Confucius said: “Don’t do to others,
what you don’t want done to you.”
In the West, this has been reframed
into a directive, more commonly known
as the Golden Rule:
“Do unto others as you would have
them do unto you.”
Instant Karma’s gonna get you
Gonna knock you right on the head
You better get yourself together
Pretty soon you’re gonna be dead
What in the world you thinking of
Laughing in the face of love
What on earth you tryin’ to do
It’s up to you, yeah you…
Lennon
Test not, that you be not tested.
For with what tests you give,
you shall be judged.
And with what measure you mete,
it shall be measured to you again.
Don’t do to others,
what you don’t want done to you…
Karma only gets you in the East.
It rarely gets you in the West.
The people who wreaked havoc on the schools with Common Core, testing , VAM, teacher firings, teacher suicides and all the rest got off Scott free.
They not only will never pay a personal price for what they did, they are still being paid handsomely to continue to tell lies.
We don’t have karma. We have truckma. Agricultural region and all that.
Ha ha ha
And presumably tracktorma and combinema too
I guess it’s spelled tractor.
Shows how much I know.
ATVma
In Texas, Florida, Wisconsin and some other states, it’s called ARma.
Tractorma and combinema would probably fall under farma.
Farma
Farma will get you
A kick from a cow
A gore from a bull, too
It’s farma, and how
Be kind to your livestock
Be kind to your wife
Be kind to your duck flock
Or run for your life!
Tougher teacher-evaluation systems can work, Petrilli said—but there was no political will to act on the results at the time of the reforms. Teachers’ unions resisted firing teachers who received poor results, and districts were unwilling or unable to pay great teachers more, he said.”
For all Petrilli’s colleagues: it must be a shear to delight to work with a fellow who would undoubtedly deny the obvious and say “I’m fine” even if he were being run over by a freight train.
Districts cannot fire their way to “success.” Unless a district knows that a teacher is beyond help, a better way that can salvage a teacher is to offer the teacher a mentoring program that can address the areas that need work. Struggling teachers need guidance and assistance before anyone lowers the boom. Not every person is a natural born teacher, but good teachers can be developed with training and patience. It takes a willingness to change, grow and do hard work to develop the skills to do the job more effectively. I was a better teacher at the end of my career than I was when I first started, and I think this is true for many teachers.
Petrelli is being handsomely paid to deny the existence of freight trains.
Very handsomely. It pays BIG TIME to be a Vichy collaborator with Gates-driven education deform.
“Tough Self-Appointed InstaEduPundit evaluation systems can work,” Bobilli said–“but there was no oligarchical will to act on the results at the time when it was OBVIOUS that test-based evaluation and school grading didn’t change anything but did cost billions of dollars and billions of hours of instructional time. The Oligarchical Covens resisted firing Self-Appointed InstaEduPundits, and Bill and the Waltons, being generous sorts but incapable of admitting how wrong they had been, were unwilling to stop the great river of green,” he said.
Billy Boy pays the bill,
so that Petrilli
can shill and trill
his now familiar,
toadying song,
about how Bill
is never wrong.
Not to mention the edTPA which has been largely sidelined by the pandemic. However, the cost incurred (in both time and money) has been astronomical and has probably prevented many would-be educators from pursuing a teacher credential. Plus, schools of education adapted to the edTPA requirements so that many pass rates are equivalent to what they were before the institution of said metric. The biggest complaint I had (among many) was that it took teacher training out of the hands of the faculty of schools of education by requiring students to focus on the edTPA (a 50-70 document) and then send it off to an unknown evaluator.
Can thank Linda Darling-Hammond for edTPA.
Among her other darling contributions to the deformation of U.S. schooling.
I guess VAM is “scientific” because it’s methods were used to gauge the production of milk in dairy cows and well, because we share 80% of our genes with cows…..guess that’s enough?!
https://www.factchecker.in/humans-cows-share-80-genes-as-home-minister-said-but-mice-dogs-apes-are-closer/
Isaac Newton (father of modern physics) ” if I have seen farther than others, it is by standing on the shoulders of Giants”
William Sanders (father of VAM, who applied methods developed for improving livestock to teachers): “If I have seen farther than udders, it is by standing on the horns of cattle”
It’s the good and great teachers who have decided they’re giving up and following the rules until they qualify for their pension. And it’s the good and great teachers who never enter the profession in the first place. It’s a pretty big universe and growing.
Also from the article:
“Even so, there are bright spots in teacher-evaluation reform, many say, most notably in Washington, D.C. The district’s teacher-evaluation system, known as IMPACT, ties student test scores to teachers’ job security and paychecks. Under the system, teachers who receive “ineffective” scores are subject to dismissal, and teachers who score “minimally effective” or “developing” could face dismissal if they don’t improve. “Highly effective” teachers, however, are eligible for financial rewards and professional opportunities.
Research has found that lower-performing teachers in the District of Columbia school system are more likely to voluntarily leave than their higher-performing counterparts. When they leave, they are replaced by teachers with higher IMPACT scores, and student achievement increases. And when they do stick around, their performance tends to improve.”
How are the Foreign Language teachers evaluated in DC? Is there a standardized test that all of their students take? If not, how does one reconcile not having standardized subject matter test scores to use versus those that do? What about bonuses? Who gets them and how is that determined?
Duane
Isn’t it obvious that foreign language teachers should be evaluated based on the standardized test math scores of their students?
And that bonuses should be based on the standardized English test scores of their students?
And that the formula used for evaluation should look like this?

Good one!
Where theta, beta, x, y, z, r, q, L and Q are pretty much whatever the hell you want them to be to get rid of the bad teachers.
And e is the base of the natural logarithms (2.71828…) and “i” is the square root of -1, of course
The latter two have no impact on the evaluation score, of course, but are just thrown in there to make the formula look more sciencey and hence more impressive.
One of those variables is often race. Think about that. Raising expectations my foot.
You are assuming that one or more of those letters actually means something.
What reason do you have for believing that that is true?
The formulas are filled with ridiculous variables to try to predict what the score should be, and one of those variables is race. So, if you are a Black person, your score is predicted to be lower than that of your white next door neighbor. It’s not just nonsense. It’s racist nonsense. Artificial intelligence is discriminatory not by accident, but by design, by programming. Not a bug, a feature.
Mathematician Cathy O’Neil (aka Mathbabe) asked to review the details for one such VAM and was rebuffed by the company that produced the computer code to implement it.
Others have had similar difficulties getting the code to figure out what, precisely, the VAM algorithms actually do, which may be something quite different from what is claimed.
These things are effectively black boxes and teachers and others have been told to simply “Trust us. We are experts who know what we are doing. Besides, it’s all very mathy and you wouldn’t understand it anyway.”
Secrecy of methods and calculations is the very antithesis of real science.
It’s a red flag for charlatanism whenever and whereever it is encountered.
The teachers in Houston filed a lawsuit alleging that no one understood the VAM algorithm, no one knew how they were evaluated. The judge agreed with them and ordered the city of Houston to stop using this inscrutable method.
Luckily , one doesn’t even have to look at the details to know the models are junk.
One can just look at the outputs, some of which have been (mistakenly) made accessible on the web.
Gary Rubenstein (a mathematician and teacher) analysed VAM data from NYCity schools and did a series of posts pointing out the complete insanity of the output.
Just one example of that can be viewed here
https://mathbabe.org/2012/03/06/the-value-added-teacher-model-sucks/
Unfortunately, Gary’s series no longer seems to be available on his WordPress blog. Not sure why that is.
As Cathy points out, VAM is basically a random (VAMdumb?) number generator.
Hell, Excel has one of this built in. Might as well just use that.
In fact, for all we know, that may be what the VAMmers have been using.
Eye opening.
The use of such black boxes to make critical decisions that affect the careers and livelihoods of teachers, with teachers having no way of inspecting or challenging the results is something right out of “The Trial”, by Franz Kafka.
The Teacherscope
I see them in the asteroids
And see them in the stars
I see the teach disasteroids
With teacherscope that’s ours
A telescope for teachers
With VAM lens on its mount
That magnifies the features
Of worst, without a doubt
That’s just BS. My sister taught (30+ yrs) MS special ed. Her evaluations became based on her students test scores. Her students were never able to pass the Alg I and ELA SBAC….tell me how that was fair to her or good for the students? She retired so that her pension wouldn’t be affected by her VAM score. She would rather bag groceries at the Walmart (if she ever needed to) than substitute teach in a district that disrespected her and her teaching skills.
LisaM,
When you say “That’s just BS”, do you mean the entire article is just BS or do you mean that the parts of the article you disagree with are BS and the parts of the article you agree with are the gold plated truth?
You know deformers like Bill Gates have achieved their primary goal (eliminating teaching as a profession) when retired teachers won’t even set foot in the schools to work as substitutes.
Mission Accomplished!
Mission accomplished!
Send in computers
Teachers are vanquished
Bots are our suitors
Contracts are written
For soft-ware and hard-
Teachers were bitten
By Gates and his guard
Do you blame the abused/disrespected teachers? My sister was quite bitter for a while after she retired upon the realization that she lost so much time with family because of her profession and the pile on of mandates and other “crap”. She gave her all and they always wanted more. My 2nd child attends a private HS and many of the teachers are former public school teachers who took buy out incentives to retire and then got a new gig in a school that is very much like public school in the late 70’s-early 80’s. They are not paid as well, but they are beyond happy with how they are treated as professionals. Win-win for my child……when teachers are happy and have the autonomy to teach appropriately, the kids are happy to go to school. I will not apologize for sending my child to a school where he is mentally healthy and learning…..mission accomplished for me as he is now looking to go to college instead of trying to find ways to stay away from school.
We can sit and argue this all day, but in the end, we all have to admit that what is going on in public education is not good for the kids and it’s not good for the teachers and staff. The only way to start the slaying of the beast is to get rid of the CC curriculum and it’s evil twin the dreaded, stupid test.
Blame the teachers?
Not I.
I am also a former teacher who would not work as a sub these days no matter how much I was paid (which is usually not much, at any rate)
Test scores do not reflect the quality of instruction; they reflect parental income and education levels. The “low performing” teachers who were driven to quit, retire too early, and in at least one case commit suicide were people who cared, wanted to make a difference in students’ lives, did make a difference in students’ lives, contributed greatly to what was once a flourishing democracy, and were beaten down by the racial and economic discrimination of standardized testing. They were not replaced by better teachers. They were not replaced. There is a terrible teacher shortage. The Seattle school district had to shut down completely on November 12th due to a lack of teachers. The University of California rejects the SAT. The UC is right. It is twenty years past time to do away with high stakes testing. There is no upside to it. None.
Leftcoastteacher,
The faculty of the University of California argued that the SAT should not be rejected, pointing out that SAT scores are a BETTER predictor of student performance at UC schools than high school grades. See this report from the faculty: https://senate.universityofcalifornia.edu/_files/underreview/sttf-report.pdf
But if they get rid of the SAT, what will people use to impress their guests at cocktail parties 50 years after high school?
There is research the shows a teacher’s influence on standardized test scores is only between 4 and 16%. Teachers can have an impact, but it is nowhere near what the deformers unsubstantiated claims. Besides, scores mean little in the real world. What matters more is a work ethic, honesty, resilience, patience, kindness and compassion.
TE,
What are you trying to say? The UC got rid of the SAT. It wasn’t unanimous, but what of that? The decision was an impactful one deliberated, based on factual information presented to the Regents and the president, by the great public university system.
I’d bet that former Theranos CEO Elizabeth Holmes had great SAT scores. After all, the average SAT for those accepted at Stanford where she was an undergrad is over 1400.
And she was clearly brilliant, managing to convince people to invest billions of dollars in a company based on a technology that ranks right up there with the Wizard of Oz’s giant yelling head.
She’s clearly the type we want at our best schools.
Leftcoastteacher,
My point is not that there was disagreement, my point is that the faculty of the University of California looked into the use of the SAT in admission and recommended that SAT scores should be used in admission decisions.
I recommend reading the faculty report.
Evaluation Session | Bob Shepherd
OK, you are sitting in your year-end evaluation session, and you’ve heard from every other teacher in your school that his or her scores were a full level lower this year than last, and so you know that the central office has leaned on the principal to give fewer exemplary ratings even though your school actually doesn’t have a problem with its test scores and people are doing what they did last year but a bit better, of course, because one grows each year as a teacher–one refines what one did before, and one never stops learning.
But you know that this ritual doesn’t have anything, really, to do with improvement. It has to do with everyone, all along the line, covering his or her tushy and playing the game and doing exactly what he or she is told going all the way back to Bill Gates, who did stack ranking because, admit it, he was probably on the spectrum and didn’t know better and, since acting in this way made him incredibly wealthy, it must be right, huh? And, at any rate, everyone knows that the tests are not actually valid or reliable and that’s not really the issue at your school because, the test scores are pretty good because this is a suburban school with affluent parents, and the kids always, year after year, do quite well.
So whether the kids are learning isn’t really the issue. The issue is that by means of the latest magic formula pushed by the district or by some InstaEduPundit, each cohort of kids is supposed to perform better than the last–significantly better–on the tests, though they come into your classes in exactly the same shape they’ve always come into them in because, you know, they are kids and they are just learning and teaching ISN’T magic. It’s a lot of hard work. It’s magical, sometimes, but its’ not magic.
So, the stuff you’ve been told to do in your “trainings” (“Bark. Roll over. Sit. Good Boy”) is pretty transparently teaching-to-the-test because that’s the only way the insane demand that each cohort will be magically superior to the last as measured by these tests might possibly be met, but you feel in your heart of hearts that doing that would be JUST WRONG–it would short-change your students to start teaching InstaWriting-for-the Test, Grade 9, instead of, say, teaching writing. And despite all the demeaning crap you are subjected to, you still give a damn.
And you sit there and you actually feel sorry for this principal because she, too, is squirming like a fly in treacle in the muck that is Education Deform, and she knows she has fantastic teachers who knock it out of the park year after year, but her life has become a living hell of accountability reports and data chats to the point that she doesn’t have time for anything else anymore (she has said this many times), and now she has to sit there and tell her amazing veteran teachers who have worked so hard all these years and who care so much and give so much and are so learned and caring that they are just satisfactory, and she feels like hell doing this and is wondering when she can retire.
And the fact that you BOTH know this hangs there in the room–the big, ugly, unspoken thing. And the politicians and the plutocrats and the mendacious, meritricious wonks at the Thomas B. Fordham Institute and that smarmy know-nothing David Coleman and the Secretary of the Department for the Privatization of Education, formerly the USDE, and the Vichy education guru collaborators with these people barrel ahead, like so many drunks in a car plowing through a crowd of pedestrians.
The teachers’ unions should file a class-action lawsuit for at least one trillion dollars to be divided among every teacher that was a victim of those torturous teacher evaluations.
The focus should have been on finding ways to move children and their families out of poverty while instilling a passion for reading in all children.
Right, Lloyd. Every teacher who was fired because of VAM should file a lawsuit for unjust termination.
The very few legal cases that have been brought regarding VAM have had outcomes that were very successful for teachers.
A class action suit that involved a large number of teachers could probably attract some of the best class action lawyers in the country because the potential monetary damages could be astronomical.
I agree too.
“Tougher teacher-evaluation systems can work, Petrilli said—but there was no political will to act on the results at the time of the reforms.”
Tougher ed-reformer evaluations can work, too – but there is no political will to shrink their bloated salaries and outsized influence over education policy.
No accountability for the accountabilitarians! Yeah, nothing we said panned out.
“But that’s because we need to stay the course for it to work!” [Note to Billy G: CCing you on this. Please send another big check.]
Accountability is for Proles.
Particularly, female Proles.
Yes, Andrea. Sadly.
Female proles = proladonnas
and surely the same could be said about politicians…
I know of no evidence whatever that “tougher teacher evaluations can work.” None. If the point is to fire teachers, who would do that during a national teacher shortage?
We simply must fire the lowest 5-10% of teachers every year until every teacher in each school is above the average for that school.
If Petrilli were evaluated for intellectual growth, he’d show a negative trend for the past two decades.
I was a teaching professional in the Michigan Public Schools when our GOP dominated state government decided that our jobs and salaries should be influenced by our student’s scores on tests that were meaningless to them. These tests had no effect on their grades and the GOP Legislature failed to fund the state’s promised scholarships for stud4ents who were found “proficient” in all subjects.
Then they passed a bill that would have required me to take state test to affirm my knowledge of the subject matter I was teaching [history, political science and economics] despite I had a permanent certificate with teaching majors and minors in each individual discipline, a masters in history and a second masters of education. I was also teaching the maximum allowed hour as an adjunct professor at our local Community College. I appealed this requirement to the State Board of Education and was told that I would be required to take this state test and would be required to take a.test preparation class as well at my own expense and with out any compensation for the hours of my time it would require. My school district was offering an excellent retirement incentage so I decided to retire early and just continue to teach for the local community Colleges.
The obsession with constant testing with high stakes for the teachers and the testing regimen required of teacher education programs has discouraged students who would have been the latest generation of wonderful teachers from entering the profession. The GOP Legislatures have gotten their real goal, they have weakened the once powerful teachers unions and are now using the teacher shortage that they created to end educational certification requirements for the warm bodies they want to put in the public school classroom who will. Work for minimum wages. This will assure that our once high quality public school systems will fail and their donors publicly funded for profit private schools [charter schools] will be the only schools in the nation.
Diane, I’m sure you don’t remember meeting me when you gave a lecture in Calvin College’s January series while you were still supporting the “charter school experiment.” But I’m sorry to say I was right when I said that the people who were behind the movement at that time in Michigan, Betsy and Dic DeVos Jr. were doing leading it as a way to weaken or destroy the MEA and the Public Schools. Unhappily, I was correct.
Kenneth Kolk,
You were right and I was wrong.
Diane, I often think that the people who should be asked how to improve educational outcomes are the teachers in the classrooms. Too often professors of education try to poll all the “stake holders” and end up relying heavily on central office administrators, educational researchers who rely on data and approach education using an industrial model. X number of students put into the system, Y amount of financial support time T amount of time should equal a uniform output at the end of the time period. Teachers know that not a single student is an uniform “input”. They all learn differently, they all learn at different rates, and the more students a teacher is expected to teach in a set time the poorer the level of educational success.
We have spent billions on Charter Schools {which usually do poorer than the public schools they draw students from}, on paying consultants to come an in-service teaching staffs on the latest methods which are far too often old methods hidden in new vocabulary. (Paradigm instead of view-point).
We would have been more successful in getting the desired educational results if we spent all that wasted money on cutting class sizes, paying teachers like the professionals they are, treating teachers as professionals and recognize them as the community assets they are.
What is an interesting outcome from the Covid Pandemic is that a lot of parents have discovered how hard their kids teachers actually work and how hard it is to actually teach their kids what they need to know. In 37 years of teaching I which I had a dollar for every time I or my fellow teachers were called dumb, lazy, incompetent; how many times I was told that I had such an easy job with so much “vacation”. If I had just gotten $1 for each time and invested it in my 403b I would be a multimillionaire.
Clearly, the study referenced in Diane’s post did not follow The First Principle of Totalitarian Education Science: “Find out what conclusions the Bill Gates Foundation for Computerizing Prole Education wants you to reach, then reach those conclusions.”
Recommended remedial instruction for the study’s authors: The Dummies Guide to Servile Meretriciousness
So, what’s the patient’s treatment regimen?
Well, we’ve been giving him daily supplements of sugar water and lead. And poking him randomly with needles and applying leeches.
And the results?
No change, though he has been complaining about the needles and leeches.
Excellent. Stay the course.
a loud guffaw
According to something I read, the suicide rate among 10-14 year-old kids tripled between 2007 and 2017. I guess I am focused on that today, as we have lost a 12 year-old recently in that most horrible of ways. And I know that correlation is not causation. Still, I cannot but wonder if the dehumanizing of school by the constant testing and public whining about it is not a part of the picture.
School is not a safe place if a student is constantly exposed to his own weakness. It should be a place where failure is common but not traumatic. Who can ever immediately find success? But surely failure should not define children they way it presently does for so many kids.
No words equal to this loss, Roy. But this is pertinent:
Tragically, conversation seems to center on how he was different, outcast, and marginalized. All of this is true. He was all of those things. But he was another casualty of a system without forgiveness. One that used to allow children to drop out instead of killing them with their own failure. We used to think that dropping out was failure, but for many students it was a sort of relief. The occasional student found economic success in dropping out. Sort of reminds you of Bill Gates.
But today restrictions are placed on students who leave school that sort of create their person as a criminal. Not that a dropout is a good thing. But a dropout is better than a suicide, if that is all you have to compare.
I thought your who’s woods these are parody was on point. What are the children doing? Is there a place at the table for all of them? Perhaps there should be.
I’m sorry for your loss. Suicide rates increasing in children should be a HUGE red flag. On top of all the deforms going on in school, social media has really kicked in with it’s own hell-scape for tweens/teens/young adults. TikTok, Instagram/Finstagram (Facebook products), Reddit/subreddits, Tumblr, Deviant art and anime sites are truly hellish places for youth and parents don’t even realize that their children (yes children!) have gained access. The kids know how to override parental controls on their pocket computers that they are attached to 24/7. Believe me, I have had to dive into some of this dark stuff and it is absolutely horrifying.
Chetty et al concluded that better teachers caused students …to graduate more frequently”
It sounds fairly far-fetched that Chetty et al concluded that better teachers would cause students to graduate more than once, but knowing some of the other stuff they concluded, I can’t be sure and wouldn’t bet the farm on its not being true, at any rate.
More than once from high school, at least.
In other news, another day, another school shooting in the United States with mass casualties. But given that this is LITERALLY a quotidian occurrence and has been for many, many years, I suspect that this one will barely register.
But by all means, let’s stay the course with our gun laws, just as we are staying the course with federally-mandated standardized testing, the Common [sic] Core [sic] and its disease variants around the country, school evaluation by test scores, teacher evaluation by test scores, and 3rd -grade retention by test scores.
It does register with the parents of the children murdered or maimed of course. Let’s not forget that.
This takes me back to many years ago in a small rural community where a student took the life of a rival in some disagreement with a rifle. It was an idyllic Tennessee town like mine, some 70 miles west. For a time everybody had the name of the shooter on their lips. We don’t want to have a ______ (his name), they said. Then there was Columbine, then more. No one recalls their names now, there seems always one more.
Don’t give a damn about “gains in student achievement’ especially when that refers to the standardized testing malpractice regime.
Duane, there were no improvements on any metrics, including graduation rates and college entry. The whole multi-billion mess was a fiasco that ruined many teachers’ lives.
When the “metrics” are pseudo-metrics with no basis in rationo-logical thinking, well. . . wasted monies are the least of our worries–as pointed out by Roy up above.
If we had adopted the metric system in the US years ago, we wouldn’t have these continuer problems.
Plus, we wouldn’t have to look up how many cups are in a quart every time we bake a cake.
We defeated the English in battle in 1783 and they are nonetheless still whipping us over 200 years later every time we make cookies.
In my state and district teachers are let go all the time because the law allowing teachers to earn a continuing contract was changed. There has been a devastating effect on morale and community schools where parents looked forward to seeing the same teachers year after year and having their younger children have some of the same teachers as their older ones. The money spent on these tests abd evaluation systems could have been spent in raises, better computers, smaller classes abd professional development.
In my state and district teachers are let go all the time because the law allowing teachers to earn a continuing contract was changed. There has been a devastating effect on morale and community schools where parents looked forward in tge past, to seeing the same teachers year after year and having their younger children have some of the same teachers as their older ones. The money spent on these tests and evaluation systems could have been spent in raises, better computers, smaller classes and professional development.
A classic Diane blog and comments. Gotta bookmark it.
Agreed. A great read for anyone who cares about, well, anything other than stock prices.
Hey Bill you do know that every school district in the country will soon fail to meet the NCLB testing requirement of 100% proficiency in math and ELA.
Of course I do David – I love it when a plan falls into place. Over 13,000 school districts and not one of them can get ever single child to pass their tests! Millions of kids all across the country are struggling with their state standards and companion tests. I love the smell of leverage in the morning David.
What say we use that leverage to force schools to adopt new “state standards”, and lets double down on the, the, ” rigor”. Who’s gonna say that they are against rigor. And while we’re at it, let’s make the tests harder than ever, I mean extra rigorous.
Great idea David! Those new Common Core standards you wrote in ELA are perfect, very subjective, lots of soft, unteachable, bogus 21st century thinking skills. Just right for objective MC tests. Ha!
The NCLB act was for losers Bill, never put any pressures on teachers to teach. It only punished districts for failing to meet AYP. Let’s link test scores directly to teacher evaluations. That should clear out the deadwood. Over 3 million teachers and more than half of them are lazy, incompetents. Who’s going to argue that good teaching shouldn’t produce good test scores! Ha!
Just one point David, we only test students in grades 3 to 8, and only in math and ELA.
What about all of the other teachers? The K to 2? The 9 to 12? The science, history, foreign language, music, art, phys ed, and others? How do we evaluate them?
Not our problem Bill.
Thanks to the war on public school teachers, I have nothing but contempt for these self-proclaimed, self-appointed, self-annointed “reformers” and their boosters (Rhee, Duncan, Brown, Cuomo [buh-bye], et al) who risked nothing and did nothing more than sit behind some friggin’ desk or ensconced themselves in some office or boardroom overlooking a park with the other self-congratulatory chuckleheads in the billionaire coffee clatch to create BS make-work that sucks the life out of real teachers when they never even had the balls to be one, let alone stay one, themselves. I have an innate, intrinsic, blood-curdling hatred of education reformers. They have enriched themselves at the expense of hardworking professionals and poverty-stricken children. Any idea for “reform” should always be looked upon with suspicion as a money grab, which the War of ’09-’17 was all about.
Actually , what matters most to succeed in our society are dishonesty and the ability to manipulate, exploit and step on (if need be) other people (you know, the traits that deformers have)
Unfortunately, most teachers don’t teach and encourage those traits because they
themselves have just the opposite mentality and the traits are therefore completely foreign and repugnant to them.
I’m retired now, but I have to say what improved my teaching all the way through my career were great teachers who were mentors and wirkshops presented by great teachers. I paid attention to my students’ test scores, but tried not to let them interfere with what I knew were best practices. Toward the end of my career, that became more difficult as administrators became more concerned about test scores.
Also, my best teaching experiences involved moving with students for 2 to 3 years. I was able to be more connected with them and their families and better able to track their progress.
Tuning in late… read every single post with pleasure. Terrific conversation here, folks. Kudos.
How many teachers lost their careers becuase union leader Randi Weingarten failed to defend teachers and took Bill Gates money to inact these reforms. Look at the damage that was done to education and students across America due to the lack of intergrity of Randi Weingarten.
Yes, we are tired of being blamed for society’s ills. My dream job has become a nightmare.