Laura Chapman, a regular contributor to the blog, has worked in arts education for many years.
She writes:
This desire to churn the teaching workforce is not just a push from Bill Gates and lawsuits to dismantle unions.
Six economists/statisticians brought together at the Brookings Institution offered a similar plan. These number crunchers said that district-wide VAM (value-added) scores should be used to determine the most effective teachers, irrespective of the subjects and grade-levels they teach.
This proposal is efficient and absurd. It is based on the assumption that a district’s value added scores are so highly correlated with “non-value added” measures that employment decisions for all teachers can be based on the performance of teachers with value added scores.
Under this system, all teachers would also have a composite evaluation based on multiple measures such as end of course test scores, observations, and student surveys. Even so, the teachers with VAM scores would determine the employment fate of all teachers. How is this conclusion reached?
Here is the magical thinking: “For example, we would assume that the correlation between observationally-based ratings of teachers and value-added (scores) in math would be the same in history, where value-added measures are not available.”
In other words, the statisticians freely invent (impute) a missing metric for the history teacher by assuming a math teacher’s rating on a classroom observation protocol can be used as a substitute for the history teacher’s missing value added score.
Those inferential leaps are just the beginning of a larger plan that would make all teacher evaluations “comparable” without any distinctions in grade level, or subject, or conditions under which teachers work.
The Brookings policy articulates principles for dismissing up to 25% of teachers in a district, on the assumption that this action plan would increase test scores and be “fair” to every teacher. The only exception to this formula might be for teachers of exceptional children. This case of econometric thinking ignores the educational, ecological, and substantive importance of different job assignments. See Corerelation, Para 5 in http://www.brookings.edu/research/reports/2011/04/26-evaluating-teachers
The Brookings paper is not radically different, (except for the 25 % churn) from a USDE plan for all teachers by a collective VAM for a school, but limited to one of the “priority” subtests such as reading or mathematics. In Florida, for example, the school wide VAM in reading or math is assigned to art and other teachers of nontested subjects. In other words, the curriculum and instruction that really matters is narrowed to the three R’s.
The use of a collective VAM focused on reading or math is a rapid and cost-effective way to meet federal or state requirements for teacher evaluation. Moreover, in 2014, a U.S. district judge ruled that evaluators in Florida are allowed to disregard a teacher’s job assignment in rating performance. The judge ruled that this practice is legal, even if it is unfair.
Teacher ratings based on a collective value-added score are likely to increase in states where Common Core State Standards (CCSS) are adopted and tested. The CCSS call for all teachers to improve student proficiency in English Language Arts and mathematics.
Although the American Statistical Association has denounced the practice of using VAM for rating individuals, that measure is unlikely to disappear as a tool for churning the workforce.
In the Obama/Duncan/McKinsey & Co. “RESPECT” project, for example, a teacher can only be judged “highly qualified” by producing more than a year’s worth of growth (gain in test scores) in three out of every five years. Teachers without that designation have shorter up-or-out criteria to meet.
This stack-ranking system, like the Brookings plan, banishes job security and churns the teaching workforce by insisting on one-size-fit-all criteria for “effective” teachers. http://www.ed.gov/blog/2012/02/launching-project-respect/

Laura ~
Who needs facts about teaching & learning, or statistics, psychometrics, or knowledge about real humans when they have bias, personal agendas & unresolved endless hatred toward teachers. Their piles are so high and deep that nothing will influence them except more money & power. That leaves teachers out completely. Teachers are the Built-in US Punching Bags to make all these sick widget counters feel better temporarily. Need constant fix. Pow! Pow! Pow!
Teachers spend their time ducking punches, draping and protecting children while posting endless meaningless data for those number crunching PoliSci major Wonkers!
Wonder what Freud would have said. Not Hysteria but Testyria?
LikeLike
The irony is that Microsoft has abandoned the stacked ranking system. Why? Because it was a complete disaster. VF Contributing Editor Kurt Eichenwald conducted extensive interviews (See Vanity Fair article “Microsoft’s Lost Decade”) – “Eichenwald’s conversations reveal that a management system known as “stack ranking”—a program that forces every unit to declare a certain percentage of employees as top performers, good performers, average, and poor—effectively crippled Microsoft’s ability to innovate. “Every current and former Microsoft employee I interviewed—every one—cited stack ranking as the most destructive process inside of Microsoft, something that drove out untold numbers of employees,”
But for some reason, Bill Gates not only ignores the evidence of what happened at his own company, but insists on spending billions to inflict this flawed system on the public education system of the United States of America. Because he believes he is right, and because he has the money so he can.
LikeLike
“Narcissism”
Narcissism holds the sway
In politics and everyday
Those who hold themselves in awe
Simply can’t admit self-flaw
LikeLike
Keep ’em coming.
LikeLike
So is Brookings planning to implement this evaluation system for their own employees, including the developers of this policy?
I will admit that I would favor a stack ranking plan for Congresscritters and other politicians, except I’d get rid of the bottom 50%. Or maybe 75%.
LikeLike
Why stop there? 99%
LikeLike
Some states are taking a more developmental and cautious approach toward using the new evaluation systems for high stakes decisions.
Wisconsin is a local control state, but asks districts to consider some important questions before utilizing the new evaluation system independently for high stakes personnel decisions. Very different from the Brookings plan.
Click to access HumanResourceConsiderations.pdf
LikeLike
Nice analysis Laura.
“Here is the magical thinking: “For example, we would assume that the correlation between observationally-based ratings of teachers and value-added (scores) in math would be the same in history, where value-added measures are not available.”
As your piece so nicely points out, magic is NOT reality. Rather it is illusion based on misdirection combined with slight of hand and often a slick delivery.
The metaphor is fitting and appropriate.
LikeLike
Wow, who these supposed experts on K12 teaching and learning processes that stand in judgement of K12 teachers?
Michelle Croft: Never taught a day of K12 in her life.
Steven Glazerman: Never taught a day of K12 in his life.
Dan Goldhaber: Never taught a day of K12 in his life.
Susanna Loeb: Never taught a day of K12 in her life.
Stephen Raudenbush: Never taught a day of K12 in his life.
Douglas Staiger: Never taught a day of K12 in his life.
Grover Whitehurst: Never taught a day of K12 in his life.
So I should give them the time of day to read their idiological bullshit????
I just took too much of my life’s time looking up to see if they ever taught K12.
The anger in me when I read idiots, many with impressive looking CVs, who refuse to listen to logical, rational arguments that destroy their conclusions and recommendations, and are so blinded by their idiology that they can’t see beyond the hand that feeds them. Surely they are the Bestest and Brightestest!!! Idiologues every last one of them!
LikeLike
Amen brother Swacker!
The only problem with our profession is that we’re mad as hell, but we keep on takin’ it.
LikeLike
Thanks for the additional disclosures about the contributors to this idiotic plan.
LikeLike
The chutzpah, the hubris, the shamelessness, the audacity, and the impudence of these folks are so far beyond my comprehension and empathetic abilities as to be . . . . . . . literally beyond words.
Have they no sense of humility?!?!
Have they no sense of propriety to not stick their noses where they have no business being?!?!
Have they no sense of “walking is someone’s shoes”?!?!
Have they no sense of humanity?!?!
Have they no sense?!?!
LikeLike
Duane~
I share your outrage, especially when I research these ‘I play a teacher 4 $M’ policy wonks and journalists who feel absolutely qualified and entitled to TELL EDUCATORS ABOUT EDUCATION!
I encourage everyone within blogger-space and Twitter-time, to research the educational and work history of the EducEggspurts.
When did this pollination & seeding of poli-wonks occur?
In test tubes?At Harvard? With turkey basters? Sorry…
Education is crawling with these folks who must have gotten the same memo – RealTeachers did not get. Too busy educating children.
There is Gold in them thar education hills!
Go West young PoliSci majors & spout your ignorance to Gates & Co. All these Institutes of Eggspurts, tweeting to challenge us & throw junk science in our faces.
Fordham sends our educational suggestions of 100 great books of children’s lit, silly skits, photos of retreats with rifles, videos related to instruction…what a cushy job & insanity.
We could not make this s*** up if we wanted to.
All about $$ & Modern Manifest Destiny.
Worked so well the 1st time! Ask the Native Americans.
LikeLike
Reformer’s Creed
Propriety or property, Rheephormers will not tell
Speculate and postulate until reform is done
Failure does not phase them, they just pack and run
Profit’s all that matters, they’ll take it straight to H$$$
How is my progress coming, DAM poet?
LikeLike
Hey, I thought Old Teachers couldn’t be taught new tricks?!?!
LikeLike
The Marzano and Danielson teacher evaluation rubrics account for 60% of our overall rating score here in NY. They are, like most of the crap produce by ed deformers who scream for evidence based change, untested and unproven. And in a similar vain, these rubrics apply to all K to 12 teachers using the same broad brush stroke. In my district, the Marzano framework for teacher evaluation is used to rate the kindergarten art teacher and the AP calculus teacher using the exact same rubric. Seriously?
LikeLike
Ask to see the reliability and validity studies of these measures with a full breakout for every grade and subject and special population. Do not accept “validity measures tied to VAM or to correlations with student surveys. If your district has rolled out the CCSS, ask to see the updated reliability and validity references.
Ask to see the contracts that your district (or state) signed that entrusted these venders with providing a metric suitable for rating your work as a teacher. What “evidence” did they use in making these high stakes decisions?
Also understand that the thrust of federal policy and state policy is standardization and with a singular concept of “effective” teaching–so generic that there is no need to consider differences in grade levels and subjects and the students in your classes.
There is lip-service to “constructivist” ideas in these protocols, but where the rubber meets the road in scoring, the exemplary teacher is the sage on the stage who delivers instruction, who solicits answers, who lets kids toss answers around a bit, who monitors them closely for “misunderstandings,” who “differentiates activities,” and always closes with an assessment that yields data for analysis, and so on.
Practices honed in the “direct instruction” movement always win in this rating game. The unstated philosophy of education calls for a completely mapped curriculum within extremely clear “learning prgressions” and curriculum that in “vertically calibrated” year-to-year.
No side trips, no fortuitous teachable moments, not much wonderment. Not much time for activities that make learning intelligible, indelible, and so powerful what you learned and how you learned it will surface in memory more than once before you die.
LikeLike
Laura
We need to hear more from you regarding the use of SLOs and SGOs. Any studies debunking this nonsense?
LikeLike
Yes a paper on SLOs (SGOs, SMART GOALS, etc) is in the works. It examines four recent USDE funded research reports and the USDE marketing program for this farce. Distribution of the paper will be via listserves of the National Art Education Association. Some excerpts from pervious work have appeared on this blog.
State officials in Maryland and the Baltimore Teacher Union recently signed on to this convoluted practice that has no peer-reviewed research to support it. The SLO process is a means to micro-manage the work of teachers as if they are sales workers who are required to tell the boss they will increase sales by a fixed amount within less than six months, and will increase the sales particularly for customers who have not been making purchases at the same rate as the majority of customers have. This is the “differentiated target” bit.
LikeLike
Laura,
Is there anyway to get a hold of your writings? I would love to be able to copy (if okay with you) and hand out to many.
Duane
LikeLike
ChapmanLH@aol.com
LikeLike
SLO’s are still relatively new and there is still quite a bit of variation in the SLO process from state to state and district to district. New York and Maryland are examples of a very standardized and inflexible approach.
There are some important choices to consider when designing an SLO process.
Are SLO’s reviewed for outcome only, or for outcome and process?
Does the evaluator approve the SLO or is the educator free to choose their SLO?
Is the student population dictated or is the educator free to choose whether the SLO includes all students or some students?
What evidence of learning qualifies? Must it be a standardized assessment, a test, course based assessments, performance rubrics?
Is there an opportunity to revise the SLO during the year?
Is scoring solely up to the evaluator or does the educator have a role in SLO scoring?
With the policy emphasis on incorporating student outcomes into evaluation systems, SLO’s are often a significant element of the evaluation system even though they are still in a developmental stage. As you note, the research base is thin and a lot will be learned in the next few years. This is yet another reason for districts and states to avoid making high stakes personnel decisions on these new evaluation systems alone.
A decade ago my state changed the licensure system where renewal is based on completing a professional development plan where the educator documents their professional growth toward a goal and how that contributed to student learning. The teachers in this licensing tier seem to find SLO’s less of a leap. The same is true for veteran teachers who chose an action research evaluation option if their district offered that.
I happen to think SLO’s have potential to improve collaboration and teaching if they are not standardized and provide teachers with choices. I have to say, though, that it is still to be demonstrated that SLO’s can be done on a large scale with the benefits are greater than the investments. I hope so, but it isn’t proven yet.
LikeLike
Screw SLOs. . . . into the beam making sure that there is some loc-tite on the threads so that they’ll never come out without burning the beam.
LikeLike
SLOs are not relatively new. They are a version of the failed business practice known as MBO promoted by Peter Drucker in 1954. The practice set up a competitive workplace and rewarded people who gamed the system. SLOs are a convoluted process of stack rating teachers in pay for performance plans, with the first version tried in Denver in 1999.
Variations in the SLO process do exist but the promoters of SLOs (paid for by USDE grants) want the most rigorous forms of SLOs so evaluations based on VAM and SLOs are comparable. Only two peer-reviewed studies on SLOs were found in a recent search of the literature also funded by USDE. One reported on SLO use in Denver, the other in Austin.
Your comments sound like they come straight from the PR machinery for SLOs funded at $43 million by USDE. The SLO process is a pseudo-scientific writing assignment for teachers. It is already required in at least 27 states. In almost all of these states an SLO must be approved by the principal or a district level evaluator. A complete SLO meets about eight criteria—description of students, baseline data about them from the prior year or pre-tests, course content and goals, choice of assessments, expected achievement levels and rates of learning pretest to posttest (euphemistically called student growth).
The preferred practice is not to permit revisions of the SLO during the year and to let a computer calculate the teacher’s effectiveness rating based on the approved measures. If there is a midcourse conference with the principal or evaluator, the purpose is not to rewrite the SLO but to chart improvements based on mid-term exams or the equivalent.
I know of only one state where the SLO process has been structured to permit a componet of teacher ”self-scoring” Tennesse. In that case it is a means to establish congruence in ratings on digital portfolios of student work. The other ratings are provided indepentely by two masked raters with special training and job-alike experience.
SLOs are used both as a strategy to standardize and micromanage the work of teachers and to focus relentlessly on student outcomes as evidence of a teacher’s effectiveness. In 15 states SLOs count for 50% of a teacher’s evaluation.
SLOs are not in “a developmental stage.“ It is disingenuous to imply they are a benign when they are being promoted as the major alternative to VAM for the majority of teachers (up to 70%) who have job assignments not tied to statewide tests. SLOs and VAM are both unreliable and invalid means of rating the effectiveness of teachers.
A reader has said that “This is yet another reason for districts and states to avoid making high stakes personnel decisions on these new evaluation systems alone.” This is a cavalier comment that sounds like it is straight from the marketing of SLOs by USDE.
The other claims made by paid marketers of SLOs are these: The SLO process encourages collaboration, improves instruction and that meas that the SLO process improves student learning. The spin masters want you to ignore the fact that there is no research to justify these claims, NONE, not even in four recent publications from one of USDE’s research arms–the Institute of Education Science!!!
The SLO process is part of the top-down management system being imposed on teachers by federal policy, and to state policy and then to district policy.
SLOs are not just another version of rationalizing plans for units and courses. State plans do differ, but the USDE bean counters want more calibrations, trainings, audits, standardization, and proofs of “statistical rigor” in calculating SLO scores for teachers. This statistically contrived “rigor” helps district generate a “spread” in ratings and set the cut scores that will classify teachers on a reductive four to six point scale ranging from “exceeds expectations” to “ unsatisfactory.”
You say “I happen to think SLO’s have potential to improve collaboration and teaching if they are not standardized and provide teachers with choices. I have to say, though, that it is still to be demonstrated that SLO’s can be done on a large scale with the benefits are greater than the investments. I hope so, but it isn’t proven yet.”
You need to do some homework. SLOs are not about choice and collaboration and evidence. They are about high stakes evaluations and how to do a soft sell of the same snake oil as VAM.
I suggest a close reading of Engaging Educators, A Reform Support Network Guide for States and Districts: Toward a New Grammar and Framework for Educator Engagement and A Toolkit for Implementing High-quality Student Learning Objectives 2.0, both available at http://www2.ed.gov/about/inits/ed/implementation-support-unit/tech-assist/resources.html
LikeLike
“The Brookings policy articulates principles for dismissing up to 25% of teachers in a district, on the assumption that this action plan would increase test scores and be “fair” to every teacher. ”
Does Brookings articulate principles for dismissing up to 25% of their own staff in order to increase the quality of their own thinking?
LikeLike
Wouldn’t take much “to increase the quality of their own thinking”!
LikeLike
Speaking of VAM…..
VAM: The Scarlett Letter
http://youtu.be/dfMymU86Bjo
LikeLike
Andy Goldstein~ VAM YouTube.
“Captain, my Captain!”
We must be our own Captains and never abandon ship.
LikeLike
https://www.youtube.com/watch?v=VylPOr7SphU
LikeLike
My problem is that I cannot follow the Marzano rubric and continue to get excellent test scores. I’ve been told for many years what to teach, but now we are being ordered how to teach it. It is almost against the law now for a teacher to go to the dry erase board and explain the strategies to solve a proportion. That is called direct instruction, and it is a bad word in my district, thanks to the Marzano model of the teacher evaluation system.
I was actually told by my evaluator that I needed to teach the highest kids in my classroom how to properly solve a proportion – and then they would teach the rest of the class in small groups, of course. It is my job to just walk around the classroom and look up at the ceiling (facilitate their learning they call it) and, of course, TEST LIKE CRAZY! I can’t teach this way! It has all gotten so ridiculous that I can’t stand the stress anymore. I love my students dearly, but all they have is a kind teacher with dark circles under her eyes with a sad smile on her face looking at the calendar on her desk to see the next assessment deadline. They deserve so much more.
LikeLike
Sad Teacher: with Señor Swacker’s kind permission, I will affirm without reservation that the quality of the heart of a caring teacher is worth more, much much more, than any quantity of test score numbers. [With a nod to Noel Wilson, natcherly…]
😄
And never forget that Mother Teresa had folks like you in mind when she said:
“Not all of us can do great things. But we can do small things with great love.”
Thank you for all you do.
😎
LikeLike
KrazyTA, God Bless you for your kind comments. I will remember your encouraging words as I work with my students this school year. What would we do without Diane’s blog? I am so thankful that I found it! It helps to know that others are experiencing the same things that I am. Thanks again!!!! (:
LikeLike
Sad Teacher, one purpose of this blog is to connect teachers and parents who find the “reforms” puzzling, frightening, absurd. To let you know you are not alone. I will be gratified the day you change your tag from “sad teacher” to “resolute teacher” or even BadAss Teacher.
LikeLike
Thank you so much, Diane! I will let you know when I feel confident to change my name! I love your blog so much. It has helped me cope. I am so grateful to you. You have been the voice for teachers all over the United States. It’s awesome to have a friend like you. (:
LikeLike
Sad Teacher, I want you to be happy, free to teach. We will all push back. They are back tracking even now, changing the tone, pretending to want a conversation, asking for a fresh start.
LikeLike
My heart goes out to you. You are in an untenable position. Either way you lose. You teach your way, the way you know works, and they fire you. You teach their way, and they fire you or undermine your belief in yourself. All I can say is document everything you do. Be prepared to present your case of “best practices.” You are a seasoned professional. No canned checklist can capture the complexity of what you do.
My story is getting real old, but I tell it again to forwarn you. Three years ago, I lost my job teaching special education at the high school level. My department chair said she would hire me in a minute to teach English, but my attempt to add material to a reading program to better fit the needs of my students (they saw as demanding a rigid prepackaged script) was seen as a big “no no.” She was instructed to let me go over her own objections and the objection of another senior department supervisor. (Interestingly enough, someone read my rebuttal letter. Several improvements were made to the program administration the following year in response to my rebuttal. Although these improvements were beyond my ability to provide, the lack of these improvements was used against me in my annual review. So much for administrative accountability.) I still struggle with the loss of that position, but I don’t think I could teach under such toxic conditions again even if my age did not work against me. I still miss the students.
LikeLike
This post was not very well written and is a bit convoluted, but I hope you know I am rooting for you.
LikeLike
Sad Teacher ~ such absolute nonsense!
Trickle Down Math? Trickle Down Education?
How’s it working for us?
Children & parents have struggled with Math Education for years, not because we are stupid, incapable of learning Math or have Dysgraphia…
Long history of new, new, new, new way of teaching Math.
Whole Language was all the rage – back to 100% phonics!
But, in Math we want our kids to reason, discover, create, conclude, collaborate without direct instruction? GenEd kids struggle and SpEd kids are COMPLETELY LOST & zone out. Agree, children need to think, process & reason…but, do they need to discover and reinvent concepts & skills?
I read an interesting article about Bill Gates watching scientific lectures while on his treadmill (oh, my what a pic). He hired that prof to appear in video Science lectures for science classes. What? We are not asking students to reinvent, discover & reason E=mc2? Only took Einstein a lifetime.
Painful process watching such nincompoops dictate Education!
Are we trapped in a reality TV show and no1 told us?
LikeLike
H.A. You brought back lots of memories with the Whole Language! Oh my gosh…how could I ever forget that time in my teaching career? My husband, who was an elementary principal at that time in history, was just talking about Whole Language the other evening. Poor Dick and Jane…they were thrown out the school window…when they were working just fine! My husband hopes that this VAM, charter school movement, teacher evaluation system, and everything else driving educators crazy will disappear like Whole Language did!
Yes, it’s sad that the Marzano rubric looks down on direct instruction.
I never thought I would see the day that a teacher is looked down upon for using direct instruction in Math. Using direct instruction is worth it for me. They can go ahead and mark me down, because I know that my students need a step by step approach to learn their Math concepts. My test scores are excellent, and I can’t get those results by aimlessly wandering around my classroom while my students teach one another.
The Common Core is much harder to teach, so I will let you know how it goes by the end of this school year. I am very nervous. My students are expected to learn concepts that I never even learned in college. It’s all crazy!
LikeLike
“HURLEY HITS NAIL ON HEAD”
Most kids want nothing to do with the constructivist/discovery approach. They are instinctively correct, as usual. This is especially true in math, science, and history.
How direct instruction ever got a bad name is beyond me. Direct instruction is only boring if the teacher is. Why any teacher buys into the “guide-on-the-side” crap is mystifying. It has been discredited in a number of studies. The discovery method is slow, inefficient, and often require the teacher to de-construct mis-learned material.
LikeLike
Be careful about wishing for efficiency: that is the basis of the scripted lessons and on-line learning modules… and identifying the “inefficient teacher” is the rationale for all of this desire to quantify.
LikeLike
NY Teacher~ When accuracy and exact correct answers are required, teachers must have the professional freedom to choose the methods. Many opportunities for creative and engaging discussions. I have nothing against them. However, dictating and hovering over teachers by the CCSS Police is absolutely insane. Heil ……!
Nothing new there, except the DICTATORIAL CCSS methods.
My concern for many children is that once they learn the incorrect answer, process, math computations, historical facts, it is difficult, especially in Math, to literally reprogram them to use the correct ways. Parents are not able to help them, even with PhDs in Math. That should tell us something!
Many LD kids, once going down the wrong path, their thinking appears to default to that. Errors from then on.
Flipped classrooms are often a disaster for those LD kids. Garbage in, garbage out and no help at home. Parents are lost and encouraged to come to training meetings at schools. WHAT? ARE WE KIDDING? Something so convoluted and total crap must cut into family time. I guess it has harmed home life already.
The beauty of credentialed and experienced teachers, we know what to do…and when!
Not anymore! Gates knows Best?
LikeLike
This is a very long copy-and-paste of info referenced by Sarah Littman above, to whom I proffer a hearty thank you!
Stack ranking? Forced ranking? Rank-and-yank? Burn-and-churn? Where have we heard that before? What oh what does it all come to in practical terms?
[start quote]
Eichenwald’s conversations reveal that a management system known as “stack ranking”—a program that forces every unit to declare a certain percentage of employees as top performers, good performers, average, and poor—effectively crippled Microsoft’s ability to innovate. “Every current and former Microsoft employee I interviewed—every one—cited stack ranking as the most destructive process inside of Microsoft, something that drove out untold numbers of employees,” Eichenwald writes. “If you were on a team of 10 people, you walked in the first day knowing that, no matter how good everyone was, 2 people were going to get a great review, 7 were going to get mediocre reviews, and 1 was going to get a terrible review,” says a former software developer. “It leads to employees focusing on competing with each other rather than competing with other companies.”
When Eichenwald asks Brian Cody, a former Microsoft engineer, whether a review of him was ever based on the quality of his work, Cody says, “It was always much less about how I could become a better engineer and much more about my need to improve my visibility among other managers.” Ed McCahill, who worked at Microsoft as a marketing manager for 16 years, says, “You look at the Windows Phone and you can’t help but wonder, How did Microsoft squander the lead they had with the Windows CE devices? They had a great lead, they were years ahead. And they completely blew it. And they completely blew it because of the bureaucracy.”
[end quote]
Link: http://www.vanityfair.com/online/daily/2012/07/microsoft-downfall-emails-steve-ballmer
See another longer piece in Vanity Fair for the following month with even more details. Under a section entitled “The Bell Curve”:
[start quote]
By 2002 the by-product of bureaucracy—brutal corporate politics—had reared its head at Microsoft. And, current and former executives said, each year the intensity and destructiveness of the game playing grew worse as employees struggled to beat out their co-workers for promotions, bonuses, or just survival.
Microsoft’s managers, intentionally or not, pumped up the volume on the viciousness. What emerged—when combined with the bitterness about financial disparities among employees, the slow pace of development, and the power of the Windows and Office divisions to kill innovation—was a toxic stew of internal antagonism and warfare.
“If you don’t play the politics, it’s management by character assassination,” said Turkel.
At the center of the cultural problems was a management system called “stack ranking.” Every current and former Microsoft employee I interviewed—every one—cited stack ranking as the most destructive process inside of Microsoft, something that drove out untold numbers of employees. The system—also referred to as “the performance model,” “the bell curve,” or just “the employee review”—has, with certain variations over the years, worked like this: every unit was forced to declare a certain percentage of employees as top performers, then good performers, then average, then below average, then poor.
“If you were on a team of 10 people, you walked in the first day knowing that, no matter how good everyone was, two people were going to get a great review, seven were going to get mediocre reviews, and one was going to get a terrible review,” said a former software developer. “It leads to employees focusing on competing with each other rather than competing with other companies.”
Supposing Microsoft had managed to hire technology’s top players into a single unit before they made their names elsewhere—Steve Jobs of Apple, Mark Zuckerberg of Facebook, Larry Page of Google, Larry Ellison of Oracle, and Jeff Bezos of Amazon—regardless of performance, under one of the iterations of stack ranking, two of them would have to be rated as below average, with one deemed disastrous.
For that reason, executives said, a lot of Microsoft superstars did everything they could to avoid working alongside other top-notch developers, out of fear that they would be hurt in the rankings. And the reviews had real-world consequences: those at the top received bonuses and promotions; those at the bottom usually received no cash or were shown the door.
Outcomes from the process were never predictable. Employees in certain divisions were given what were known as M.B.O.’s—management business objectives—which were essentially the expectations for what they would accomplish in a particular year. But even achieving every M.B.O. was no guarantee of receiving a high ranking, since some other employee could exceed the assigned performance. As a result, Microsoft employees not only tried to do a good job but also worked hard to make sure their colleagues did not.
“The behavior this engenders, people do everything they can to stay out of the bottom bucket,” one Microsoft engineer said. “People responsible for features will openly sabotage other people’s efforts. One of the most valuable things I learned was to give the appearance of being courteous while withholding just enough information from colleagues to ensure they didn’t get ahead of me on the rankings.”
Worse, because the reviews came every six months, employees and their supervisors—who were also ranked—focused on their short-term performance, rather than on longer efforts to innovate.
“The six-month reviews forced a lot of bad decision-making,” one software designer said. “People planned their days and their years around the review, rather than around products. You really had to focus on the six-month performance, rather than on doing what was right for the company.”
There was some room for bending the numbers a bit. Each team would be within a larger Microsoft group. The supervisors of the teams could have slightly more of their employees in the higher ranks so long as the full group met the required percentages. So, every six months, all of the supervisors in a single group met for a few days of horse trading.
On the first day, the supervisors—as many as 30—gather in a single conference room. Blinds are drawn; doors are closed. A grid containing possible rankings is put up—sometimes on a whiteboard, sometimes on a poster board tacked to the wall—and everyone breaks out Post-it notes. Names of team members are scribbled on the notes, then each manager takes a turn placing the slips of paper into the grid boxes. Usually, though, the numbers don’t work on the first go-round. That’s when the haggling begins.
“There are some pretty impassioned debates and the Post-it notes end up being shuffled around for days so that we can meet the bell curve,” said one Microsoft manager who has participated in a number of the sessions. “It doesn’t always work out well. I myself have had to give rankings to people that they didn’t deserve because of this forced curve.”
The best way to guarantee a higher ranking, executives said, is to keep in mind the realities of those behind-the-scenes debates—every employee has to impress not only his or her boss but bosses from other teams as well. And that means schmoozing and brown-nosing as many supervisors as possible.
“I was told in almost every review that the political game was always important for my career development,” said Brian Cody, a former Microsoft engineer. “It was always much more on ‘Let’s work on the political game’ than on improving my actual performance.”
Like other employees I interviewed, Cody said that the reality of the corporate culture slowed everything down. “It got to the point where I was second-guessing everything I was doing,” he said. “Whenever I had a question for some other team, instead of going to the developer who had the answer, I would first touch base with that developer’s manager, so that he knew what I was working on. That was the only way to be visible to other managers, which you needed for the review.”
I asked Cody whether his review was ever based on the quality of his work. He paused for a very long time. “It was always much less about how I could become a better engineer and much more about my need to improve my visibility among other managers.”
In the end, the stack-ranking system crippled the ability to innovate at Microsoft, executives said. “I wanted to build a team of people who would work together and whose only focus would be on making great software,” said Bill Hill, the former manager. “But you can’t do that at Microsoft.”
[end quote]
Link: http://www.vanityfair.com/business/2012/08/microsoft-lost-mojo-steve-ballmer
I apologize for the length of this comment. But I would add one more thing: all the above was foreseen decades ago by W. Edwards Deming.
Literally, the whole disastrous mess was predictable and preventable.
😎
LikeLike
“Literally, the whole disastrous mess was predictable and preventable.”
Just as stank racking, the educational standards and standardized testing “disastrous mess was/is predictable and preventable” but the difference being that in the mean time those educational malpractices are harming countless innocent young students who never have a say as to whether they would want to be a part of that process in contrast to the employees of Microsoft who voluntarily were part of that system.
CHOCK FULL OF EPISTEMOLOGICAL ERRORS, COMPLETELY INVALID, UNETHICAL USAGES AND IMMORAL CONSEQUENCES for the most innocent, the students, are what the current educational standards and standardized testing regime is.
And yes it was predicted by Noel Wilson, but he isn’t a god so he couldn’t have prevented those educational malpractices. But prove the invalidities in the process and predict the harmful effects he did in his never rebutted nor refuted “Educational Standards and the Problem of Error” found at: http://epaa.asu.edu/ojs/article/view/577/700
Brief outline of Wilson’s “Educational Standards and the Problem of Error” and some comments of mine. (updated 6/24/13 per Wilson email)
1. A description of a quality can only be partially quantified. Quantity is almost always a very small aspect of quality. It is illogical to judge/assess a whole category only by a part of the whole. The assessment is, by definition, lacking in the sense that “assessments are always of multidimensional qualities. To quantify them as unidimensional quantities (numbers or grades) is to perpetuate a fundamental logical error” (per Wilson). The teaching and learning process falls in the logical realm of aesthetics/qualities of human interactions. In attempting to quantify educational standards and standardized testing the descriptive information about said interactions is inadequate, insufficient and inferior to the point of invalidity and unacceptability.
2. A major epistemological mistake is that we attach, with great importance, the “score” of the student, not only onto the student but also, by extension, the teacher, school and district. Any description of a testing event is only a description of an interaction, that of the student and the testing device at a given time and place. The only correct logical thing that we can attempt to do is to describe that interaction (how accurately or not is a whole other story). That description cannot, by logical thought, be “assigned/attached” to the student as it cannot be a description of the student but the interaction. And this error is probably one of the most egregious “errors” that occur with standardized testing (and even the “grading” of students by a teacher).
3. Wilson identifies four “frames of reference” each with distinct assumptions (epistemological basis) about the assessment process from which the “assessor” views the interactions of the teaching and learning process: the Judge (think college professor who “knows” the students capabilities and grades them accordingly), the General Frame-think standardized testing that claims to have a “scientific” basis, the Specific Frame-think of learning by objective like computer based learning, getting a correct answer before moving on to the next screen, and the Responsive Frame-think of an apprenticeship in a trade or a medical residency program where the learner interacts with the “teacher” with constant feedback. Each category has its own sources of error and more error in the process is caused when the assessor confuses and conflates the categories.
4. Wilson elucidates the notion of “error”: “Error is predicated on a notion of perfection; to allocate error is to imply what is without error; to know error it is necessary to determine what is true. And what is true is determined by what we define as true, theoretically by the assumptions of our epistemology, practically by the events and non-events, the discourses and silences, the world of surfaces and their interactions and interpretations; in short, the practices that permeate the field. . . Error is the uncertainty dimension of the statement; error is the band within which chaos reigns, in which anything can happen. Error comprises all of those eventful circumstances which make the assessment statement less than perfectly precise, the measure less than perfectly accurate, the rank order less than perfectly stable, the standard and its measurement less than absolute, and the communication of its truth less than impeccable.”
In other word all the logical errors involved in the process render any conclusions invalid.
5. The test makers/psychometricians, through all sorts of mathematical machinations attempt to “prove” that these tests (based on standards) are valid-errorless or supposedly at least with minimal error [they aren’t]. Wilson turns the concept of validity on its head and focuses on just how invalid the machinations and the test and results are. He is an advocate for the test taker not the test maker. In doing so he identifies thirteen sources of “error”, any one of which renders the test making/giving/disseminating of results invalid. And a basic logical premise is that once something is shown to be invalid it is just that, invalid, and no amount of “fudging” by the psychometricians/test makers can alleviate that invalidity.
6. Having shown the invalidity, and therefore the unreliability, of the whole process Wilson concludes, rightly so, that any result/information gleaned from the process is “vain and illusory”. In other words start with an invalidity, end with an invalidity (except by sheer chance every once in a while, like a blind and anosmic squirrel who finds the occasional acorn, a result may be “true”) or to put in more mundane terms crap in-crap out.
7. And so what does this all mean? I’ll let Wilson have the second to last word: “So what does a test measure in our world? It measures what the person with the power to pay for the test says it measures. And the person who sets the test will name the test what the person who pays for the test wants the test to be named.”
In other words it attempts to measure “’something’ and we can specify some of the ‘errors’ in that ‘something’ but still don’t know [precisely] what the ‘something’ is.” The whole process harms many students as the social rewards for some are not available to others who “don’t make the grade (sic)” Should American public education have the function of sorting and separating students so that some may receive greater benefits than others, especially considering that the sorting and separating devices, educational standards and standardized testing, are so flawed not only in concept but in execution?
My answer is NO!!!!!
One final note with Wilson channeling Foucault and his concept of subjectivization:
“So the mark [grade/test score] becomes part of the story about yourself and with sufficient repetitions becomes true: true because those who know, those in authority, say it is true; true because the society in which you live legitimates this authority; true because your cultural habitus makes it difficult for you to perceive, conceive and integrate those aspects of your experience that contradict the story; true because in acting out your story, which now includes the mark and its meaning, the social truth that created it is confirmed; true because if your mark is high you are consistently rewarded, so that your voice becomes a voice of authority in the power-knowledge discourses that reproduce the structure that helped to produce you; true because if your mark is low your voice becomes muted and confirms your lower position in the social hierarchy; true finally because that success or failure confirms that mark that implicitly predicted the now self evident consequences. And so the circle is complete.”
In other words students “internalize” what those “marks” (grades/test scores) mean, and since the vast majority of the students have not developed the mental skills to counteract what the “authorities” say, they accept as “natural and normal” that “story/description” of them. Although paradoxical in a sense, the “I’m an “A” student” is almost as harmful as “I’m an ‘F’ student” in hindering students becoming independent, critical and free thinkers. And having independent, critical and free thinkers is a threat to the current socio-economic structure of society.
LikeLike
If we look at Bill Gates agenda for the use of VAM in Microsoft, we see that they plan to replace the bottom 25-percent of workers annually. This year, Microsoft is getting rid of about 23 – 24 percent of its workers, and has already asked for an increase in H1 Visas to replace them even though there are plenty of qualified unemployed or underemployed Americans in that field.
If the Gates rank and yank system is applied to the public schools, that means every four years, the entire 3.3+ million workforce of K to 12 teachers in the U.S. would have to be replaced.
When they run out of Americans to fill those jobs every four years, would they turn to India and China for recruits? What happens when they use up all the qualified English speakers in the world? Robots? Computers?
LikeLike
Wasn’t it Deming who say one of the first steps to improving a business or organization was to drive out fear? Even if there was a deep pool of qualified teachers waiting to be hired in the millions, the kind of churning some of the reformers advocate just kills trust, culture, and continuity in a school. This is harmful for both the children and the adults in the school.
That is enough reason to condemn this kind of nonsense, but it’s also the case that there isn’t this giant pool of untapped teaching talent waiting to get into schools. Any sane administrator knows that you are better off developing the teachers you have than ranking and yanking. If you have a truly ineffective teacher (not common), you work with them on a plan of intensive supervision and professional development. If they progress, you keep them and celebrate. If they don’t progress, you give them due process and part ways. At this point, administration needs to reflect on what happened in the hiring and support process.
LikeLike
“Any sane administrator knows that you are better off developing the teachers you have than ranking and yanking.”
From what I’ve read, this is exactly what the countries that have the top performing public education systems are doing—even Communist China. None of them, so far, are using the Bill Gates rank and yank method of public education destruction.
LikeLike
You both are right on the mark with your comments. The evaluators at my school told the staff at the beginning of last year that our evaluations would fall on the bell curve. The evaluators in many schools are not fair, of course, and so they are deciding who falls where on this curve. If you are a favorite and on their committees (you have to be asked by them), you will score very well. If you are an older teacher, you will be chosen to fall on their low end. It is all a big game which destroys teachers’ self esteem. I’ve never seen so much discouragement in my life. There are favorites who run out of the school building at 3PM who have higher ratings than teachers who are still in their classrooms at 6PM. I would have had to retrain into another profession. I would have never survived this craziness for 30 years.
LikeLike
The premise for all of this quantification and standardization is that “bad teachers” are the problem and the causes and conditions that students bring to school are immaterial. Economists are notorious for creating mathematical models are precise and exacting but do not measure what is important.
LikeLike
If I may correct your statement a bit:
“Economists are notorious for creating mathematical models [THAT SUPPOSEDLY AND/OR PURPORTEDLY] are precise and exacting but do not measure what is important.”
LikeLike