Education is being destroyed by data-driven decision-making. The algorithms make no sense. VAM doesn’t correctly identify teacher quality. The essence of good teaching cannot be reduced to a number. The metrics are fraudulent. Big data misleads. People cannot be treated as widgets.
Now David Brooks is saying these things about our politics.
He writes:
“Unfortunately, the whole thing has been a fiasco. As politics has gotten more scientific, the campaigns have gotten worse, especially for the candidates who overrely on these techniques.
“That’s because the data-driven style of politics is built on a questionable philosophy and a set of dubious assumptions. Data-driven politics is built on a philosophy you might call Impersonalism. This is the belief that what matters in politics is the reaction of populations and not the idiosyncratic judgment, moral character or creativity of individuals.”
Just substitute the words “education” or “schooling,” and the same points are valid. Now if only one of the Néw York Times daily columnists would see the parallels.
Diane Ravitch's blog 
Reblogged this on David R. Taylor-Thoughts on Texas Education.
Well, that is because JUNK SCIENCE was used RE: CCSS and high-stakes testing.
It’s as though garbage-in-garbage-out is immaterial now that there is a lucrative market for garbage.
Even a blind chipmunk finds a nut sometimes.
The anosmic one also!
I thought one of the causes (I had to look it up) relevant to the data-driven discussion:
the loss of the sense of smell, either total or partial. It may be caused by head injury, infection, or blockage of the nose.
Mitchell… in light of “RECENT CURRENT EVENTS”… data-driven politics combined with money-driven politics = DISASTER. Robert Reich has a great and telling “tale”…
http://robertreich.org/post/101613921835
Does anyone know of any resources (ungated journal articles, blog posts, etc.) that explain precisely what’s wrong with value-added scores? I’m looking for something that actually gets into the mathematical weeds and explains this stuff in detail, not just something along the lines of “THE RHEEFORMER$$$ DON’T REALIZE THAT SOMETHING SO ARTFUL AND SUBTLE AS TEACHING CAN’T BE CAPTURED BY SOMETHING AS VULGAR AS A *NUMBER*”
Check out http://vamboozled.com/research-articles-on-vams/
Might want to check out Gary Rubinstein’s blog posts on VAMs (here, for example)
Here’s the link Diane posted on this site to the statement issued by the American Statistical Association. http://www.amstat.org/policy/pdfs/ASA_VAM_Statement.pdf
The best study of the invalidities of the processes of educational standards and standardized testing is Noel Wilson’s “Educational Standards and the Problem of Error” found at: http://epaa.asu.edu/ojs/article/view/577/700
Brief outline of Wilson’s “Educational Standards and the Problem of Error” and some comments of mine. (updated 6/24/13 per Wilson email)
1. A description of a quality can only be partially quantified. Quantity is almost always a very small aspect of quality. It is illogical to judge/assess a whole category only by a part of the whole. The assessment is, by definition, lacking in the sense that “assessments are always of multidimensional qualities. To quantify them as unidimensional quantities (numbers or grades) is to perpetuate a fundamental logical error” (per Wilson). The teaching and learning process falls in the logical realm of aesthetics/qualities of human interactions. In attempting to quantify educational standards and standardized testing the descriptive information about said interactions is inadequate, insufficient and inferior to the point of invalidity and unacceptability.
2. A major epistemological mistake is that we attach, with great importance, the “score” of the student, not only onto the student but also, by extension, the teacher, school and district. Any description of a testing event is only a description of an interaction, that of the student and the testing device at a given time and place. The only correct logical thing that we can attempt to do is to describe that interaction (how accurately or not is a whole other story). That description cannot, by logical thought, be “assigned/attached” to the student as it cannot be a description of the student but the interaction. And this error is probably one of the most egregious “errors” that occur with standardized testing (and even the “grading” of students by a teacher).
3. Wilson identifies four “frames of reference” each with distinct assumptions (epistemological basis) about the assessment process from which the “assessor” views the interactions of the teaching and learning process: the Judge (think college professor who “knows” the students capabilities and grades them accordingly), the General Frame-think standardized testing that claims to have a “scientific” basis, the Specific Frame-think of learning by objective like computer based learning, getting a correct answer before moving on to the next screen, and the Responsive Frame-think of an apprenticeship in a trade or a medical residency program where the learner interacts with the “teacher” with constant feedback. Each category has its own sources of error and more error in the process is caused when the assessor confuses and conflates the categories.
4. Wilson elucidates the notion of “error”: “Error is predicated on a notion of perfection; to allocate error is to imply what is without error; to know error it is necessary to determine what is true. And what is true is determined by what we define as true, theoretically by the assumptions of our epistemology, practically by the events and non-events, the discourses and silences, the world of surfaces and their interactions and interpretations; in short, the practices that permeate the field. . . Error is the uncertainty dimension of the statement; error is the band within which chaos reigns, in which anything can happen. Error comprises all of those eventful circumstances which make the assessment statement less than perfectly precise, the measure less than perfectly accurate, the rank order less than perfectly stable, the standard and its measurement less than absolute, and the communication of its truth less than impeccable.”
In other word all the logical errors involved in the process render any conclusions invalid.
5. The test makers/psychometricians, through all sorts of mathematical machinations attempt to “prove” that these tests (based on standards) are valid-errorless or supposedly at least with minimal error [they aren’t]. Wilson turns the concept of validity on its head and focuses on just how invalid the machinations and the test and results are. He is an advocate for the test taker not the test maker. In doing so he identifies thirteen sources of “error”, any one of which renders the test making/giving/disseminating of results invalid. And a basic logical premise is that once something is shown to be invalid it is just that, invalid, and no amount of “fudging” by the psychometricians/test makers can alleviate that invalidity.
6. Having shown the invalidity, and therefore the unreliability, of the whole process Wilson concludes, rightly so, that any result/information gleaned from the process is “vain and illusory”. In other words start with an invalidity, end with an invalidity (except by sheer chance every once in a while, like a blind and anosmic squirrel who finds the occasional acorn, a result may be “true”) or to put in more mundane terms crap in-crap out.
7. And so what does this all mean? I’ll let Wilson have the second to last word: “So what does a test measure in our world? It measures what the person with the power to pay for the test says it measures. And the person who sets the test will name the test what the person who pays for the test wants the test to be named.”
In other words it attempts to measure “’something’ and we can specify some of the ‘errors’ in that ‘something’ but still don’t know [precisely] what the ‘something’ is.” The whole process harms many students as the social rewards for some are not available to others who “don’t make the grade (sic)” Should American public education have the function of sorting and separating students so that some may receive greater benefits than others, especially considering that the sorting and separating devices, educational standards and standardized testing, are so flawed not only in concept but in execution?
My answer is NO!!!!!
One final note with Wilson channeling Foucault and his concept of subjectivization:
“So the mark [grade/test score] becomes part of the story about yourself and with sufficient repetitions becomes true: true because those who know, those in authority, say it is true; true because the society in which you live legitimates this authority; true because your cultural habitus makes it difficult for you to perceive, conceive and integrate those aspects of your experience that contradict the story; true because in acting out your story, which now includes the mark and its meaning, the social truth that created it is confirmed; true because if your mark is high you are consistently rewarded, so that your voice becomes a voice of authority in the power-knowledge discourses that reproduce the structure that helped to produce you; true because if your mark is low your voice becomes muted and confirms your lower position in the social hierarchy; true finally because that success or failure confirms that mark that implicitly predicted the now self evident consequences. And so the circle is complete.”
In other words students “internalize” what those “marks” (grades/test scores) mean, and since the vast majority of the students have not developed the mental skills to counteract what the “authorities” say, they accept as “natural and normal” that “story/description” of them. Although paradoxical in a sense, the “I’m an “A” student” is almost as harmful as “I’m an ‘F’ student” in hindering students becoming independent, critical and free thinkers. And having independent, critical and free thinkers is a threat to the current socio-economic structure of society.
By Duane E. Swacker
Cathy O’Neil, aka “Mathbabe,” has written numerous posts on VAM, and published the writing of others.
Try searching for “The Value Added Teacher Model Sucks,” from her “Mathbabe” blog.
David Brooks. Nailed it.
Reblogged this on The Buzz and commented:
Oh David, it’s not such a big leap to apply this pholosphy to education, is it?
Let’s not forget the impossible goals of NCLB! In politics candidates don’t really suffer when the pollsters make a claim that comes up short. This is untrue in the privatization movement where students and their families suffer, careers are destroyed and public schools, which rightly should be part of the public trust, are starved while greedy CEOs line their pockets with taxpayer money.
The cynicism of the W Administration was to make the goals become particularly impossible once he was long gone from office. He also added in a loaded phrase: “the soft bigotry of low expectations.”
I like the more accurate phrase:
“The hard bigotry of impossible requirements”
Bush was an expert on the bigotry of low expectations, especially when it came to himself (Mission accomplished!)
And Gates (an expert on “the software bigotry of low expectations”) seems to have continued the trend with VAMs: “The statistical bigotry of low correlations”
NY Teacher: I am not trying to top you, but as I occasionally wrote on this blog, the phrase “the soft bigotry of low expectations” was replaced by “the hard bigotry of mandated failure” during the era of NCLB/RTTT [aka NoChild’sBehindLeft/DashForTheCash].
SomeDAM Poet: krazy props. And we don’t even have to wait ten years to have a 98% “satisfactory” [double thanks, Mr. Bill Gates!] chance of certainty that your VAManiacal jab was deftly delivered.
A painful paradox: we are living in a time when impossibly unrealistic goals are proposed & mandated by startlingly incompetent thoughtless mediocrities.
Our pain, their gain.
$tudent $ucce$$, anyone?
[start quote]
(The district on Monday also updated details of Deasy’s separation agreement. He’ll receive about $61,000 for unused vacation days in addition to about $70,000 in severance to be available as needed through year’s end.)
[end quote]
Link: http://www.latimes.com/local/education/la-me-lausd-cortines-20141021-story.html
Remembering, today, tomorrow, and forever, that’s it’s all about the kids…
😒
KTA
Mandated failure v. Impossible requirements
YOU WIN!
Meet you down at the Pink Slip.
I owe you one.
“mandated failure”
Good one! Pretty well sums it up.
also know as
“VAMGated failure”
VAMGated failure
Ed reform
BillGated flailure
Is the norm
Lets definitely not forget about the unconstitutional NCLB law. It needs to be challenged in court. A law that is IMPOSSIBLE to abide by, a law that is de-facto ENTRAPMENT, a law that every single entity (school) under its jurisdiction is FORCED to VIOLATE as a result of its impossible demands must be deemed UNCONSTITUTIONAL. This impossible requirement is the root of the test-and-punish federal regime. If NCLB goes away, so does CCSS/RTTT/APPR/VAM/PARCC/SBAC
I was excited until I learned that he wasn’t talking about education.
I voted for this person today:
http://www.voteforlymanstall.com/
I met him and liked him and (generally) agree with him on many (although not all) things but mostly it was just really nice to vote for someone who values public schools. No apologies. He backs public schools. That doesn’t mean he thinks public schools are perfect, it just means he doesn’t want them privatized. Clear, strong position. Not an “agnostic”. Agnostics make lousy advocates.
He’ll have an uphill battle. It’s tough to win as a Democrat here but good for him for trying and good for him for making public schools his campaign focus.
Classic Brooks. This column could have been written anytime in the last 50 years, with “polling” substituted for “data.”
That message needs to get to USDE and Bill Gates and their clones.
More than once I have ranted about the Gates-funded “data quality” campaign, and the USDE Student Teacher Data Link, and the survellience system being constructed to connect parental caregiving, teachers, principals, schools, school boards, and teacher education programs to “college and work-place” outcomes.
That long paper trail is way beyond the routine reporting of the National Center for Education Statistics, NAEP data, and the longitudinal data gathered from another program since the mid-1980s, and US. Census information– all publicly available.
In at least five states, teachers are writing SLOs that are being evaluated not on their merit as elaborated course plans, but by nit-picking evaluators, often ignorant of the subject and grade level but trained to inspect SLOs for compliance with 25 to 30 criteria in writing. The SLO is writing assignment , often with a computer template that forces teachers to meet the coding requirements for baseline data on all students in their classes, standards and content for the course, every “target” for learning, every instructional strategy, and every district or state approved test (with district-approved scoring guides for teacher-developed tests).
Anonymous reviewers at one of the RttT subcontractors are functioning as overseers of the detail that must be included in the SLOs making recommendations for districts and freely imposing their own views of “proper” aims and methods of teaching on students.
The efforts to introduce standardized language for the work of teachers is not just driving them crazy, but corrupting what scholars call the “language and thought” in and about education.
Teachers and others are speaking of “impacting the growth of students” as if that is a desirable thing to do.
In federal policies (RttT) “rigor” means statistical rigor … usually reduced to a “cut score.”
So hurray for David Brooks.
He is actually recycling ideas from economists who are wondering whether “big data” and the analytics churned out by software are as trustworthy as the hype. I call this trend “the econometric turn in education.” Here are some words from some experts on big data.
We are more susceptible than we may think to the “dictatorship of data”—that is, letting the data govern us in ways that may do as much harm as good. The threat is that we will let ourselves be mindlessly bound by the output of our analyses even when we have reasonable grounds for suspecting something is amiss.
Or that we will attribute a degree of truth to data which it does not deserve.
Viktor Mayer-Schönberger & Kenneth Cukier. (2013). Big Data: A Revolution That Will Transform How We Live, Work, and Think. Boston: Houghton Mifflin. p. 166.
Good points, Laura. Crunching numbers is privileged today, but there are other –often better –ways of knowing. Rebecca Newberger Goldstein’s book Plato at the Googleplex: Why Philosophy Wont Go Away does a good job of vindicating old-fashioned modes of investigating the truth. In my experience, “hard” education data rarely reveals much truth. My own experienced teacher brain, flawed as it is, is a much more sensitive and reliable instrument for gauging what’s happening in my students’ brains.
yes! more philosophy. I believe that strongly.
I am so curious to see results of elections tonight, knowing some about these campaigns being data crazy.
It’s true, what he says here.
Maybe we’re turning a corner. ??
Yeah, Illinois turned a corner alright. Unfortunately, we turned the wrong way down a one-way street.
A dark day for public education here in Illinois. Quinn was not the best candidate, but with Rauner and Emmanuel at the helm, it’s going to get real ugly, real fast. Mike Madigan will go along because he loves being a power broker and hates sharing the stage with a Democratic governor. Mike Madigan wants to muddy up Rauner so his daughter, Lisa Madigan, can run against Rauner in 2018. Our best hope, Karen Lewis, meanwhile has been diagnosed with a brain tumor.
I’m losing faith fast. The level of ignorance among downstate voters is appalling:
Consistently voting against their self-interests.
Electing a tax-dodging, job-destroying plutocrat for governor.
The 3 percent millionaire surtax (which would raise $1 billion for education in Illinois) barely passed in our county, which ranks near rock-bottom for wages in the state of Illinois
I think it’s time we challenge our IFT and IEA leadership. Our pinkie-ring loving IFT leader in Joliet is worthless, and says we basically have accept whatever “reform” gets shoved down our collective throats.
.It’s time we demand that they stop cozying up to the politicians who are destroying public education in our state. ..
The Democrats I was pushing won.
We are very happy here in Western NC about that (not about the US Senate, though).
So. . .as for the data. I think there’s probably a balance of using it to track voting and then having the candidate stand for something voters want to see.
We had record mid-term voter turnout here. I’m so happy about that. LOTS of grassroots groups were working their butts off here for at least the last nine months.
Very happy here in Western NC.
“Data Driven Drivel”
Data Driven Drivel
What it’s all about
Better get your shovel
To drive the drivel out
Except for governor, I voted Libertarian all the way, here in Illinois. And if I wasn’t a teacher my vote for governor would have been Libertarian too!!!!! It’s like there is NO choice what-so-ever. Even my vote for governor is a waste because my choice is better for teachers but worse for schools!!!
Which non-Libertarian candidate do you think is better for teachers? The one who wants to profit from privatized public goods or the one who stole pensions from teachers?
Sorry, this is a bit off point but I can’t contain myself. Tom Wolf wins in Pennsylvania. Corbett is gone, our pensions and tenure are saved, and now there is the real possibility that Pennsylvania will finally adopt a fair funding formula and Philadelphia teachers and students will get the resources they are entitled too.
Today I like David Brooks.
Brooks: “The more you look at political history, the more you see that political imagination is the rarest and most valuable of qualities. Voters don’t always know what they want, but they look to leaders to jump ahead of the current moment and provide visions they haven’t thought of.”
He’s right.
Political imagination is what’s lacking in the California Teachers Association. There’s no creativity. There’s no idiosyncratic leadership. It seems like they’re led by a team of lackluster/ smart consultants who are doing their best to ape “serious” cool corporate types. The membership needs real, warmblooded, intelligent, creative leadership. What it gets are insipid magazine articles, a few flyers with facts about the candidates, and lots of reminders about member benefits. How inspiring! Cichocki and Heins, the heirs apparent, seem like competent managers but not imaginative leaders. David Goldberg seems more promising, but I don’t know him that well.
Speaking of abusing data — VAM in Missouri (put on the ballot by rich political sugar-daddy Rex Sinquefield with the help of our soon-to-be-ex-Commissioner-of-Education Chris Nicastro) is going down! 77% of Missouri voters say “NO!” with 16% of the votes in!
77% of Missouri voters say “NO” to VAM — with 43% of the vote counted.
All the vote counted and the 77% against holds — Missouri voters give unequivocal NO to VAM!
I’m getting sick from watching the national and local election returns.
Rauner is now ahead of Quinn in IL and, at the same time, the news reported that 68% of those same people voted to increase the minimum wage to $10 per hour here. Ooops! They picked the governor who does not support that!
What on earth is wrong with people? That’s a whole lot of folks who got that one wrong. When are Democrats going to realize that the education policies they have been promoting are resulting in them losing their base to some seriously ignorant voters, many of whom don’t really care about the suffering of other people, especially if they differ from them?
Maybe more people need to suffer, so they’ll know what the rest of us have been going through. No doubt, our new Congress will now ensure austerity for the masses –and riches for the few. Just hope people learn this by 2016. Meanwhile, I feel like I did when Reagan won. And Bush. I have lost my faith in the intelligence of voters.
David Brooks, ever-sensitive to the latest-floated political meme;. Wow, so he just picked up on the public’s growing disillusionment with data-driven policy. Maybe he’ll even connect the dots– eventually [when poll results are ripe] to data-driven ed policy… Maybe, when the political winds are favorable, he’ll write an op-ed against VAM… whee.
For some reason, tonight the site asked for valid email & name. You all know me better as “Spanish & French Free-lancer
Perhaps politics and education are human endeavors that can not and should not be driven by data as compared to disciplines such as science and engineering.
Plus, what some folks (eg, those who support VAMs for firing) call “data driven” is not really “data driven” at all, but actually the reverse: “driven data”
The data are twisted (cherry picked, massaged and otherwise misrepresented) to give a predetermined outcome
Data driven
or driven data?
Latter given
Former, later!