Steven Singer writes here about the mechanistic, anti-child implicationsand consequences of data-driven Instruction. He identifies six issues. I offer only the first of these problems. To learn about the other five, open the link.
He writes:
No teacher should ever be data-driven. Every teacher should be student-driven.
You should base your instruction around what’s best for your students – what motivates them, inspires them, gets them ready and interested in learning.
To be sure, you should be data-informed – you should know what their test scores are and that should factor into your lessons in one way or another – but test scores should not be the driving force behind your instruction, especially since standardized test scores are incredibly poor indicators of student knowledge.
No one really believes that the Be All and End All of student knowledge is children’s ability to choose the “correct” answer on a multiple-choice test. No one sits back in awe at Albert Einstein’s test scores – it’s what he was able to do with the knowledge he had. Indeed, his understanding of the universe could not be adequately captured in a simple choice between four possible answers.
As I see it, there are at least six major problems with this dependence on student data at the heart of the data-driven movement.
So without further ado, here is a sextet of major flaws in the theory of data-driven instruction:
The Data is Unscientific
When we talk about student data, we’re talking about statistics. We’re talking about a quantity computed from a sample or a random variable.
As such, it needs to be a measure of something specific, something clearly defined and agreed upon.
For instance, you could measure the brightness of a star or its position in space.
However, when dealing with student knowledge, we leave the hard sciences and enter the realm of psychology. The focus of study is not and cannot be as clearly defined. What, after all, are we measuring when we give a standardized test? What are the units we’re using to measure it?
We find ourselves in the same sticky situation as those trying to measure intelligence. What is this thing we’re trying to quantify and how exactly do we go about quantifying it?
The result is intensely subjective. Sure we throw numbers up there to represent our assumptions, but – make no mistake – these are not the same numbers that measure distances on the globe or the density of an atomic nucleus.
These are approximations made up by human beings to justify deeply subjective assumptions about human nature.
It looks like statistics. It looks like math. But it is neither of these things.
We just get tricked by the numbers. We see them and mistake what we’re seeing for the hard sciences. We fall victim to the cult of numerology. That’s what data-driven instruction really is – the deepest type of mysticism passed off as science.
The idea that high stakes test scores are the best way to assess learning and that instruction should center around them is essentially a faith based initiative.
Before we can go any further, we must understand that.

“To be sure, you should be data-informed – you should know what their test scores are and that should factor into your lessons in one way or another. . . ”
Ay, ay ay ay friggin AY!
Steven explains how those scores are worthless, yet he still can’t get out of the test score mode, as if something completely invalid should have some kind of value. Hell, test scores don’t even have any value as manure. Why, oh why should a teacher know what their-the student’s test scores are? And why factor that invalid, misleading and false nonsense into one’s lessons “in one way or another”. Why?
Why?
Why?
LikeLiked by 1 person
Unfortunately, the tests are here and they won’t go away anytime soon. There is not enough of a revolt by parents yet and even if there were, we still have ESSA which binds us to test scores. It’s a vicious cycle. I think Singer is choosing a strategy……the best way to deal with the test is to use the “scores” as a way to defeat the test (and the testing companies). I would like the tests gone, too! I refuse when I can, but sadly, our state (MD) uses PARCC as a graduation requirement. I really want my one child in public school to receive her HS diploma.
LikeLike
“Unfortunately, the tests are here and they won’t go away anytime soon.”
Well considering the massive harms done to the students in the standards and testing malpractice regime. . .
They should have been long gone before they even started.
Absurdities, inaninities and injustices abound in the standards and testing malpractice regime. Why would anyone lend any credence whatsoever to such insanities? Why?
Why?
Why?
LikeLike
Duane, sure the test scores are terrible, but that doesn’t mean you shouldn’t know what they are. I often use them when I talk to administrators. For example, let’s say I know a student deserves to be in the advanced class because of authentic observations I conducted in my classroom. I can’t tell my admin that. He’d never listen to me. But if I know this student also also got a Proficient on the stupid Common Core test, I can use that to make my point. That’s the kind of thing I mean.
LikeLike
And, yes, I may know what the scores are. And any communication by me of those scores would, ethically in my mind, come with a huge caveat, a denial of the validity and therefore my usage is suspect, corrupt, not “faithful to truth”.
LikeLike
I am guilty of using test scores to convince administrators of this or that. It’s sometimes all they can understand. I do not EVER use test scores to inform my teaching methods, however. Not ever. I read the students’ essays. I read and listen to their answers to my questions. Their answers to silly Common Core style, poorly constructed, standardized, computerized, multiple choice questions are meaningless. Mr. Singer is right — test scores are invalid. It is always the correct viewpoint to take and must be applied as much as possible. The scores are bunk.
LikeLike
The deformers want JIM CROW Laws. They won’t say this outloud, but they say it anyway to those who are simpatico with them.
So many don’t understand democracy and the importance of diversity. They truly want everything to be the same … you know, kinda like growing only one crop and then having disease and then famine.
LikeLike
You know, Yvonne, that I have to disagree with you on the Jim Crow thought.
They also don’t say out loud that making a lot of money is there real intention.
I do though agree with you on the importance of diversity and not standardization!
LikeLike
imagine a world with no people, above us only sky….how does that song go?
LikeLiked by 1 person
LikeLiked by 1 person
Thanks for your clarification, Steven. Hoist them on their own petards, eh.
Unfortunately, though, what you are doing (and I understand the intention behind it) serves to reinforce the standards and testing malpractice regime even though that is not your purpose/desire at all.
Your adminimals need to be educated about those malpractices, although I think that may be a near impossibility as their jobs depend on them not knowing and/or ignoring the consequences of the standardized testing malpractice.
LikeLike
I object to the equation of “data” and “test scores”. Data includes everything that happens in (and out of) a classroom and we shouldn’t cede that word. Teachers should be data driven (and I imagine most are). They shouldn’t be test-score driven. Heck, I don’t even know how they could be test-score driven considering they don’t find out test scores unti the following year. Ignore tests altogether.
LikeLike
How about “data-informed,” rather than data driven. Nothing drives me but my interests, goals, passions, curiosity. Certainly not data.
LikeLike
How about “student work/portfolio informed”? Data are just numbers and symbols. Useful information can’t be boiled down to data. Yes, I am student informed by talking to my students and seeing their work.
LikeLike
Yeah, every decision is “data driven” or “data informed” (which I like only a little bit more), but before the realization that they could swamp teachers in numbers through cool aps, data was not a word that darkened most classroom doors. Furthermore, the word has been weaponized like “personalized learning” and “educational reform.” Our smartest move may be to continue what we have done now that the terminology has infiltrated so deeply into the conversation that people can recite it like those annoying TV jingles.. We need a fractured version of the terms, just like we have done with “depersonalized learning” and “deformers.” “Data uninformed” just doesn’t have a ring to it. Anyone?
LikeLike
Data Drivel
LikeLike
Teachers learn from assessing students, and none of this information needs to be from a standardized test. I learned more about elementary students’ literacy from a brief informal reading inventory, a running record, and a short writing sample than any information I ever gleaned from a standardized test. I often got useful information on how to help students by simply observing them. The information I gathered provided me with diagnostic information that helped me plan for the needs of each student. The information then informed my instruction, and it was directly helpful to students. That is real “personalized learning.”
LikeLike
Classroom Teachers have the “real-time” information and are able to process this “real-time.”
The deformers are so wrong. They just want to label, label, label and the labels are twisted to benefit the oligarchs for their profits.
LikeLike
Retired colleague,
Yes!
LikeLike
Exactly right. Works the same way at every level. It’s almost like dialectic.
LikeLike
Hudson: “The largest element of fakery is a category that is imputed-
that is, made up.”
The “mark and its meaning” IS MADE UP.
One final note with Wilson channeling Foucault and his concept of subjectivization:
“So the mark [grade/test score] becomes part of the story about yourself and with sufficient repetitions becomes true: true because those who know, those in authority, say it is true; true because the society in which you live legitimates this authority; true because your cultural habitus makes it difficult for you to perceive, conceive and integrate those aspects of your experience that contradict the story; true because in acting out your story, which now includes the mark and its meaning, the social truth that created it is confirmed…”
The test score “Flag” based standards…
Wilson:
“At the very least, discourse about standards will be emotionally charged. Talk of
changing educational standards is like talk of changing the flag. It triggers all the fears of
change in the social realities, be they ever so violating, for which the standard, and the
flag, are symbols.
By insisting in this thesis that educational or ability standards have no empirical reality, I
cut much more deeply into the social fabric. For such a claim not only undermines the
standard, but also by association denigrates the social reality that it represents. The
metaphor is not changing the flag, but destroying it, on the grounds that the social order
that it pretends to represent is a delusion, very different to the one that it does indeed
refer to. A delusion whose continuance, furthermore, is largely sustained through the
emotional effects of the inviolability of its recurring symbol, the flag.
The person who destroys the flag is inviting extreme social response, for such is its
emotional content that many people will identify this map with its territory. For them, to
destroy the flag is to destroy the social order it represents, and thus to destroy their
identity within that order. Emotionally, social symbol and social reality are contiguous.
For many people, this contiguity overlaps and symbol and referent become identical. In
this state of mind, cognitive arguments and empirical data have as much impact as
falling animals crashing into rocks. As much impact on the rocks, that is.
In an analogous way, to criticise the notion of educational or job standards on the
grounds that they cannot in practice be measured or logically sustained is to destabilise
the symbol of the meritocritous society, the competitive capitalist order that it supports,
and the cult of individualism that, almost alone, it defines and constructs. Emotionally,
these four constructs – standard, competition, meritocracy, and individualism, are deeply
intertwined. To threaten one of them is to threaten all. And to threaten all is to threaten
each one of us, you and I and him and her. For it is to threaten that social order in which
we all, in our own way, or more likely in a way that the structure has imposed on us, has
found our place.”
LikeLike
Finishing up reading Wilson’s dissertation for the umpteenth time (lost count well over a dozen) and as I am reading, I am constantly saying to myself “There’s a quote that is current, credible, cogent and that is unique and fundamental. I should copy and keep that thought.” Nary a page is read without that thought.
All educators should read and understand what Wilson is saying in his “Educational Standards and the Problem of Error” found at: http://epaa.asu.edu/ojs/article/view/577/700
LikeLike
“The key components of the 2018 Ohio School Report Cards.
The six components are
Achievement,
Gap Closing,
Improving At-Risk K-3 Readers,
Progress,
Graduation Rate,
Prepared for Success.
Districts and schools receive A-F grades on each of the six components and most of the individual measures for each component (e.g.a letter grade is assigned to Ohio’s EVASS metrics based on test scores. EVASS is a version of totally discredited VALUE-ADDED Metrics).
For the first time, districts and schools will be assigned overall letter grades. (e.g., Your school is D. Your school is an F.)
Here is the pitch for this ridicule-worthy scheme.
“Report cards are designed to give parents, communities, educators and policymakers information about the performance of districts and schools – to celebrate success and identify areas for improvement. This information identifies schools to receive intensive supports, drives local conversations on continuous improvement and provides transparent reporting on student performance. The goal is to ensure equitable outcomes and high expectations for all of Ohio’s students.”
One of these days I may count how many data points Ohio has shoved into the convoluted report card. Some are hardwired by the fools elected to the state house. Others are there in part from federal regulations. The rest are the product of a belief system that says, in effect measurement is an objective and infallible substitute for good judgment. Of course, the Report card grades track the relative affluence of the districts in Ohio and they are meaningless for Charter Schools. A recent conversation with a state school board member, running for re-election revealed total ignorance of problems with the value-added metric or the cost of the SAS contract for that misleading exercise.
reportcard.education.ohio.gov
LikeLike
SAD … and very BAD. Good grief, do people think anymore?
LikeLike
Yvonne,
Most useful thinking about education occurs here, on this blog.
LikeLike
I was pleased to see the word “numerology” used in this, for the term captures perfectly that “data-driven instruction” has come to mean in the era of state testing and the puerile Common Core “Standards.” This has long been my go-to term for describing our current testing regimen. It’s numerology. It’s pseudoscience. It deserves to go on the ash heap of history alongside astrology, alchemy, phrenology, and proposals for perpetual motion machines and squaring the circle.
Enough, already. Kids are being hurt by this, and in ELA, one of the many ways in which they are being harmed is through the dramatic distortion of curricula that is occurring in order to make the “standards” into a curriculum outline and instructional materials into test prep. The hair on the tail is wagging the dog.
We need to make it very, very clear when people espouse the “data-driven instruction” line that we know that this is pseudoscience. The ONLY proper response to this nonsense is derision. Those who cling to this pseudoscience need to be laughed off the national stage.
LikeLike
cs: first sentence, “what,” not “that”
LikeLike
“Enough, already. Kids are being hurt by this.”
And that fact is the biggest abomination that damn near every single teacher and administrator choose to slough off. Innocents are in their charge, en loco parentis anyone, and those innocents are being harmed.
To hell with those “educators”* who choose self-interest over justice for the children.
*and I hesitate to call anyone who knowingly implements malpractices that harm student ‘educators’. They certainly aren’t teachers.
LikeLiked by 1 person
Off the topic, squaring the circle apparently became synonymous with trying the impossible due to a Greek play and its reference to that among other impossible things.
The quadrature of the circle, the lune, and other curved geometric forms was actually the series of constructions logically allowable to draw a square of equal area to the circle or whatever. Close approximations resulted in discoveries about pi in ancient times, and produced some fascinating logic along the way.
LikeLiked by 1 person
Yes, Mr. Shepherd. Yes.
LikeLike
Do click on Mr. Singer’s article and read it in its entirety. It’s a beautiful piece.
LikeLike