Matthew Kraft, an assistant professor of education and economics at Brown University, asks the question in the title of this post. He weighs the pros and cons. The cons are overwhelming. The pros are dubious. Peter Greene assesses Kraft’s balancing act. He finds the pro side of the balance curiously vacuous. Yet Kraft steps back and concludes that the nation is better off for the millions or hundreds of millions or billions spent to evaluate teachers by dubious measures.
Greene is not buying it.
The rush to evaluate demoralized teachers, created perverse incentives, and created a teacher shortage, with no evidence of success. Not much of a positive scorecard. (By the way, whatever happened to Raj Chetty and his Nobel-worthy research demonstrating that your fourth grade teacher determines your lifetime income and whether you [if female] get pregnant before marriage?)
He writes:
“Previous sucky evaluation systems may not have provided useful information about teachers (or depended on being used by good principals to generate good data). But at least those previous systems did not incentivize bad behavior. Modern reform evaluation systems add powerful motivation for schools to center themselves not on teachers or students or even standards, but on test results. And test-centered schools run upside down– instead of meeting the students’ needs, the test-centered school sees the students as adversaries who must be cajoled, coached, trained and even forced to cough up the scores that the school needs. The Madeline Hunter checklist may have been bunk, but at least it didn’t encourage me to conduct regular malpractice in my classroom.
“So yes– everyone would be better off if the last round of evaluation “reforms” had never happened.
“Kraft also asks if “the rushed and contentious rollout of teacher evaluation reforms poison the well for getting evaluation right.”
“Hmmm. First, I’ll challenge his assumption that rushed rollout is the problem. This is the old “Program X would have been great if it had been implemented properly,” but it’s almost never the implementation, stupid. There’s no good way to implement a bad program. Bad is bad, whether it’s rushed or not.
“Second, that particular well has never been a source of sparkling pure water, but yes, the current system made things worse. The problems could be reversed. The solution here is the same as the solution to many reform-created education problems– scrap test-centered schooling. Scrap the BS Test. Scrap the use of a BS Test to evaluate schools or teachers or students. Strip the BS Test of all significant consequences; make it a no-stakes test. That would remove a huge source of poison from the education well.”

“New administrative data from student information systems that, linked to teacher human resource systems, allow administrators and researchers to answer a range of important questions about teacher effectiveness.” This is a great piece of bureaucrat-speak. “Administrative data from student information systems” mean the test results harvested from all the countless tests. And by the way, CAASP takes several days five hours a day – this is crazy even for a supporter of moderate testing like me. All this pain and fear and vomiting by kids is just to “allow administrators and researchers to answer a range of important questions about teacher effectiveness.” Damn, boy.
LikeLike
This money spent on tests could have been spent on lowering class size, improving libraries, after school programs, school nurses and counselors and quality art, music and drama classes.
LikeLike
I thought Bill Gates is in the lowering class size business already.
LikeLike
Bill Gates publicly supports larger class sizes on the theory that if all teachers are “great” teachers, as determined by VAM, then class size doesn’t matter.
LikeLike
Not in my state, anyway. Classes in my middle school can be as high as 42 in an art class and 40 in an academic class. Class sizes are the largest that I have ever seen them.
LikeLike
Knowing how he likes to have his ego stroked, I’m sure Melinda prolly reassured Bill long ago that size does not matter.
LikeLike
Even better, don’t give it at all.
It’s not only Campbell’s Law that’s involved, although it certainly is, but the way the tests are designed means they will mostly confirm over and over that whiteness and wealth equal intellectual competence.
Which wouldn’t be so bad if we concluded that we should therefore more fairly distribute wealth. And whiteness?
Deb
Sent from my iPhone
>
LikeLike
The teachers know more about their students than any “commercial” test.
Yes, scrap the BS tests.
LikeLike
Update on what’s happening in Mississippi…funding attached designed to leave with a student to school of choice so far didn’t pass but this did:
IT WORKED! — Your efforts paid off for our students! J Joyce Helmick, MAE President [https://boomerangoutlook.baydin.com/static/img/icons/ribbon/boomerang_64_blue.png]
Reply| Today, 11:14 AM You [http://can2-prod.s3.amazonaws.com/email_templates/logos/000/007/884/original/Webp.net-resizeimage_(4).png] CONGRATULATIONS!
March 26, 2018
Dear Members,
We just received word that the conference report on House Bill 1592, the K-12 funding measure, has been adopted with an increase of $12.8 million over the current fiscal year funding.
The measure includes these increases below
· $3.1-million more for the Mississippi Adequate Education Program (MAEP)
· $2.5-million more for Early Learning Collaboratives (pre-k)
· $5-million more for the School Recognition Program (teacher bonuses)
There is still $15 million allocated for literacy coaches, $12 million for teacher supply funds, and the National Board Certified Teacher program and Chickasaw Cession interest payments appear to be fully funded.
Your phone calls and outreach to legislators in your districts made all the difference! We had asked that they not give our public schools a penny less than $8.2 million over level funding (which was originally proposed), and they went above and beyond that figure.
There will be more detailed analyses on the bill reported in the media soon, and we’ll be sharing those on Facebook, Twitter and the web site when they come in, but right now, we just wanted to tell you THANK YOU! for your continued efforts during the session.
It never hurts to tell these folks we appreciate their efforts, too. So, here again is the link that will provide you with contact information on your local legislators so you can give them a pat on the back.
Stand by for a comprehensive report on the successful 2018 Legislative session. We know the real battle for full funding and letting our teachers teach is far from over, but for the moment:
HAPPY DANCE!
[https://can2-prod.s3.amazonaws.com/uploads/data/000/162/769/original/Happy_Dance.jpg]
Warmly,
[https://can2-prod.s3.amazonaws.com/uploads/data/000/162/770/original/Joyce_first_name.png]
JOYCE HELMICK MAE President
________________________________
LikeLike
Evaluating teachers is an imperfect, complex task. Greene concludes that while the old observation system was imperfect, it was far better than the data crunching new models that hinge on the big, standardized tests are more imperfect. Standardized tests waste time feeding the data monster when there are far more important issues in running a school. We already know that we cannot fire our way to success. We also know that minimally trained teaching temps do not have the same level of expertise as experienced certified teachers. As Greene aptly points out efforts to compare teachers in urban and suburban districts is akin to comparing a “runner going up a hill to a runner coming down.” It’s a false analogy; yet “reform” continues making false assumptions about teachers.
The best option for evaluation of teachers is one that is local. Local administrators understand the needs of the community and students, and they have been trained in observation protocol. Despite some potential political issues, they are also the best choice to provide assistance to those in need of improvement. If the goal of evaluation is to work towards improvement, local control is better than hanging careers on an algorithm based on false assumptions.
LikeLike
Indeed, the only option is local because that’s the only need. The only people who have any interest in how any given teacher is doing are that teacher’s supervisors (principal, assistant principal, superintendent, school board) and the taxpayers of that community. The state (much less the feds) has absolutely no business being involved in teacher evaluation. It doesn’t matter if Ms. Smith in Walla Walla is better or worse than Mr. Jones in Wichita. The only thing that matters is whether Ms. Smith and Mr. Jones are adequately educating the kids in their respective care.
LikeLike
“whatever happened to Raj Chetty and his Nobel-worthy research demonstrating that your fourth grade teacher determines your lifetime income and whether you [if female] get pregnant before marriage?)
Well, don’t write him off just yet.
He still has a chance for as long as the Swedish central bank continues to award their fake “Nobel”.
Validity is not even on the list of requirements for the award.
In fact, a few years back they gave the award to economists who contradicted each other!
Here’s William Black in “Economics could be a science if more economists were scientists”
http://neweconomicperspectives.org/2013/10/economics-science-economists-scientists.html
, which was written in response to a piece Raj Chetty wrote in the NY Times
William Black:
“This year [2013] not for the first time, the central bankers decided to hedge their bets – awarding their prize to economists who contradict each other (Eugene Fama and Robert Shiller). The hedge strategy might be thought to ensure that the central bank’s prize winners were right at least half the time (which would be an improvement over the central bankers’ batting average in their awards), but that is a logical error. It is perfectly possible for both of the prize winners to be wrong. “
LikeLike
“Poisoning the Well”
You’ve poisoned the well
With reason and fact
And now I can’t sell
The software I hacked!
LikeLiked by 1 person
“The Roarschock Test for schools”
We learned a lot
From testing blot
A Roarshock test of sorts
To see the good
You really should
Ignore reformy warts
LikeLike
“Was all the money spent on testing worth it?”
‘Twas worth it to the Pearsons
‘Twas worth it to Elia
‘Twas worth it to the persons
Who pushed their earnings higher
LikeLike
‘Twas worth it to the Duncans
Who used revolving door
To monetize their flunkins:
The tests and Common Core
‘Twas worth it to the scammers
Who took the loot and ran
The testers and the VAMmers
Who always had a plan
‘Twas worth it to the tankers
With millions from the Gates’
The thinker tanker wankers
Who charged their highest rates
‘Twas worth it to Deformers
To privatize the schools
Who Rheelly ain’t “reformers”
But privatizing tools
LikeLike
privatizing tools. great ending.
LikeLike
Maybe the strongest pro argument to test-based evaluation up to this point is perhaps learning more conclusively that it’s not worth it.
Now let’s dispense with it and move on.
LikeLike
Annual testing is not worth it, not the money, time, emotional upheaval, destabilization, ruined lives, narrowed content, or loss of student privacy, but it suggests that dismantling public school districts with any charters, online classes, tech-based instruction, vouchers, or temp teachers does not improve “outcomes”. Annual testing debunked privatization and itself at the same time! Gates doesn’t care. He wants your data.
LikeLike
Even the framing of the question is wrong
The question should not be “Was all that time and money spent on test based evaluation worth it”
The question should be “why were Bill Gates, Arne Duncan and others allowed to impose evaluation methods like VAM and SGP on schools and teachers, when the preponderance of available evidence indicated that they were not only not capable of reliably distinguishing good from poor teachers (were often no better than flipping a coin, in fact) but were likely to negatively impact the teaching and learning for millions of students?”
LikeLike
Why was the NCLB Act allowed to pass in the first place? Why was education ever blamed for putting the “Nation at Risk”, when it wasn’t, in the first place? It’s all been one great charade, a Testing Charade. Lies, all lies. Lies mingled with Ayn Randian greed, investment hubris, and a supersized seving of stupidity.
LikeLike
Peter Greene graced my early Monday morning with out-loud chuckles, all the way from the Professor of Astrophysics and Cheeseburgers to the Awesome Teacher tree in the back yard. Thank you!
LikeLike
Posted a link to Peter’s article;
https://www.opednews.com/Quicklink/Teacher-Evaluation-Plus-o-in-Life_Arts-Education-Testing_Information_Peter-Greene_School-Reform-180327-777.html#comment694766
with this comment from this blog. (embedded links at the above address
As Daniel Koretz showed in his book The Testing Charade: Pretending to Improve Schools, the tests are misused and abused to make these judgments. They aren’t good enough to label students, teachers, or schools, and their misuse distorts the measures (Campbell’s Law).
Now a group of scholars seeks to rescue schools from the iron grip of standardized testing. (Among the authors is my favorite economist of education, Helen Ladd of Duke University.) They argue that test scores are not the only things that matter in education.They say that schools should beinformed by evidence, not driven by it.
John Thompson wrote an excellent review of Daniel Koretz’s “The Testing Charade” in the Huffington Post.
LikeLike
I so hope Mr. Kraft gets to read Mr. Greene’s rejoinder. Frustrating that no comments are allowed on his propaganda piece. Surprise, surprise that he “concludes” that Mr. Kraft finds a modest but significant positive impact. He knows what he has to determine.
LikeLike
Kraft’s claim is positive all right: positively absurd.
LikeLike
“Deformer Successes”
Failures are many
Successes are few
Can’t think of any
Successes, can you?
LikeLike
“or billions spent to evaluate teachers by dubious measures.”
It’s easily billions: let’s not forget how much the testing companies get just for conduction the year end standardized tests: it’s well over $200 million in Tennessee alone. Of course the exact amount is difficult to track down.
LikeLike
The amount that a few companies like Pearson have earned from testing is just the tip of the iceberg.
The potential education “market” in the US alone is easily worth tens of billions a year, especially if “personalized” learning takes off and companies can charge $1000 or more per head (money that will almost certainly come from laying off teachers)
$1000 x 65 million is $65 billion.
People like Bill Gates certainly recognized the potential goldmine, which is why he was willing to drop a few hundred million on Common Core, which was meant to standardize schools in order to facilitate sale of software.
LikeLike