Our friend Peter Greene is now a regular contributor to Forbes, where he enlightens readers from the business community about education. Many years ago, I wrote a column for Forbes and visited their offices. I discovered to my surprise that my editor was married to a classroom teacher. We have friends everywhere.
Recently, Peter has been enlightening readers of Forbes about standardized testing.
In this one, he shows that standardized tests don’t show much that matters.
“There are plenty of reasons to doubt the validity of the Big Standardized Test, be it PARCC or SBA or whatever your state is using these days. After almost two decades of its use, we’ve raised an entire generation of students around the notion of test-based accountability, and yet the fruits of that seem…. well, elusive. Where are the waves of students now arriving on college campuses super-prepared? Where are the businesses proclaiming that today’s grads are the most awesome in history? Where is the increase in citizens with great-paying jobs? Where are any visible signs that the test-based accountability system has worked?”
Even a few reformers are doubting the value of the BS test.
In this piece, he explained why standardized testing is beyond repair.
Diane Ravitch's blog 
Another great article from Mr. Greene. Should be noted that teaches have been saying this from the beginning.
and that is years and years and YEARS now…
This is a good piece, as far as it goes. Unfortunately, in a short piece like this, it’s impossible even for one with the gifts of Peter Greene to get at the underlying issues. I’d like to talk about one of those—the nature of Educational Reform movements.
If you taught in the 1960s, ‘70s, or ‘80s, you were subject to the then-current Reform model of Behavior Modification. Education Reforms tend to be slow learners. They went whole hog for Behavior Modification during that period even though psychologists, by and large, throughout the U.S. and Great Britain, had thrown off Behaviorism in what we now refer to as the Cognitive Revolution (thanks to such pioneering work as Chomsky’s on language acquisition and Lashley’s on improvisation, both of which required positing mental operations). In schools of the time, every lesson plan was supposed to have Behavioral Objectives and to proceed according to models of classical and operant conditioning. And the preferred pedagogical model was programmed learning. The subject would be divided into tiny chunks, students would proceed through these independently and be tested for mastery before going on to the next chunk. The place where this model was most completely implemented was the Language Lab. There was just one problem—it didn’t work. Kid in language labs, using programmed learning methods, didn’t learn more Spanish or French than did kids in regular language classrooms. In fact, they learned much less. Why? Well, they were bored to tears. Contra the Behaviorists, what was going on in the kids’ heads actually mattered.
So, the Behaviorist fad, borrowed from psychology, petered out, to be replaced by the next Reform fad, analytics—being data driven—which was borrowed from business, and we’re now seeing that this magic formula for improving education is also a bust. Standards-and-testing-based data mongering has a) not improved outcomes and b) not reduced achievement gaps. It’s an utter failure by its own preferred measure—test scores.
If history is any guide—and what else can be?—when the era of this current magic formula for education has passed, it will be replaced by another. What will that be? Doubtless some other set of top-down mandates and command-and-control procedures. Perhaps it will be standardized testing based on content knowledge. But again, this will fail because the problem lies in the whole notion of centralized command and control. People function best in conditions of autonomy within which they can innovate and respond to the particular circumstances they are in based in part on tried and true heuristics, in part on new ideas deriving from the marketplace of ideas, and in part on their own new ideas. Top-down overspecification and standardization throws a wrench into this natural system. Instead of another wave of top-down Deform, what we need to do is to return to a system of autonomous local public schools in which teachers in departments make their own decisions about pedagogy and curricula. Under such a system, states would issue very broad frameworks describing general goals to be achieved, and there would be a national wiki of COMPETING curriculum outlines, model lessons and assessments, general and domain-specific vocabulary and reading lists, and other such materials, continually updated and added to by classroom practitioners, researchers, and subject-area specialists, from which local teachers, in their departments, would draw.
The whole is smarter than the individuals who dictate to everyone else. We don’t need another wave of Ed Reform by self-appointed Thought Police. We need to leave teachers free to make their own curricular and pedagogical decisions, in collaboration with members of their departments. In such conditions of autonomy, subject to that departmental social sanction and to established rules of the tribe, innovation occurs, and real improvement is made.
But the first thing to do—the most important thing—what must happen before we can move on from the current Deforms—is to throw over the national standardized testing mandate. Until that happens, we will make no progress. And I suspect that the only way to make that happen, short of letting its failure play out for another decade, is if the national teachers’ unions get behind ending the testing mandate and organize walk-outs, strikes, hearings to expose the invalidity of the standardized testing, and so on.
I taught from the mid ‘60’s through the early 2000’s….been there done that – until I somehow got a bit more wise to how my students actually learned and began engaging them in investigations in mathematics-algebra, geometry, pre calculus. Amazing how much more they learned and how many connections they were able to make…and how much more they enjoyed class! Asking them to apply that knowledge brought interesting and very positive and well thought out projects and demonstrations of what they knew and were able to show me. I am somewhat glad I am no longer teaching in public school….the testing, CCSSM, etc would drive me nuts!
We will NEVER be able to get rid of Common Core without the Standardized Testing regime coming to an end. It’s the testing that props up all the other dreadful things……test prep as curriculum, scripted lesson plans, lack of reading in ELA, wonky math skills, SEL etc. I have said many times during the Spring test season, that teachers need to boycott these tests and refuse to administer them (they need to have some guts), but I always get the same old excuse of “we need our jobs”. The focus lately has been on corruption in the Charter Industry, and I agree that it is a huge waste of tax payer dollars earmarked for education, but Charter schools only educate 10-15% of the nation’s student population. EVERY child who attends a public school or a charter school is subjected to the standardized testing. The focus needs to change and the teachers unions need to make that a priority.
Teachers will only be able to do that in large numbers with their unions behind them. The unions have to step up. Otherwise, we shall simply have to see the current deform regime play out in ever-increasing failure, waste of resources, dumbing down and bastardization of pedagogy and curricula, etc. Must we wait until another whole generation of students is robbed of a humane education?
All the testing is the will of the 1% as it is a tool of privatization. Since billionaires and corporations own many of the decision makers, it will be difficult to free public schools from the yoke of testing.
The unions are part of the problem. They have been far too complicit in working with “reformers” instead of working for teachers that pay their dues. The focus does need to change, but undoing bad policy is not easy.
“I have said many times during the Spring test season, that teachers need to boycott these tests and refuse to administer them (they need to have some guts), but I always get the same old excuse of “we need our jobs”.”
No doubt the cojoneless GAGA Good German teachers have contributed to the holocaust of the most innocent minds, those of the young students. And the NEA and AFT have been Vichy collaborators in that holocaust.
Two quotes come to mind. One from a premier American thinker, the other from a French philosopher.
From the former:
“The mass of men [and women] serves the state [education powers that be] thus, not as men mainly, but as machines, with their bodies. They are the standing army, and the militia, jailors, constables, posse comitatus, [administrators and teachers], etc. In most cases there is no free exercise whatever of the judgment or of the moral sense; but they put themselves on a level with wood and earth and stones; and wooden men can perhaps be manufactured that will serve the purpose as well. Such command no more respect than men of straw or a lump of dirt.”- Henry David Thoreau [1817-1862], American author and philosopher [my additions]
From the latter:
“Should we therefore forgo our self-interest? Of course not. But it [self-interest] must be subordinate to justice, not the other way around. . . . To take advantage of a child’s naivete. . . in order to extract from them something [test scores, personal information] that is contrary to their interests, or intentions, without their knowledge [or consent of parents] or through coercion [state mandated testing], is always and everywhere unjust even if in some places and under certain circumstances it is not illegal. . . . Justice is superior to and more valuable than well-being or efficiency; it cannot be sacrificed to them, not even for the happiness of the greatest number [quoting Rawls]. To what could justice legitimately be sacrificed, since without justice there would be no legitimacy or illegitimacy? And in the name of what, since without justice even humanity, happiness and love could have no absolute value?. . . Without justice, values would be nothing more than (self) interests or motives; they would cease to be values or would become values without worth.”—Comte-Sponville in “A Small Treatise on the Great Virtues” [my additions]
Not holding my breath until I see the unions start taking a strong, active response against the federal standardized testing mandate, but they clearly have a responsibility to do this. Only they can make it happen sooner rather than later. And, ofc, the time is ripe politically. The leaders of the two major teachers’ unions need to understand that if they did this–if they galvanized their membership to work toward ending the tests–they would have an OVERWHELMING MAJORITY of not only teachers but also of the general population behind them. Parents HATE these tests almost as much as teachers do. If these union leaders want really to revitalize the union movement AND ensure their continued relevance and power AND do a lot of good for kids, then they will make the standardized testing Enemy Number 1. We need mass action. Protests. Opt outs. Marches on state capitols. A march on Washington. And, importantly, HEARINGS to expose the tests and their publishers.
This country should learn from the Puerto Ricans. They are mad as hell and not going to take it anymore.
“Standards-and-testing-based data mongering has a) not improved outcomes and b) not reduced achievement gaps.”
You have focused on outputs with that statement. Exactly what the deformers and privateers would like us to focus on. Never have given a rat’s patoot about “improving outcomes” or “reducing achievement gaps” as doing so forces us to focus on that which is out of control of the teacher and to not devote time to one of the most important aspects of being a quality teacher, that of improving the teaching and learning process (which you go on to describe in the rest of your response) as needed for each individual student.
You are quite correct that now we must focus on getting out from under the thumb of the standards and testing malpractice regime and turn our focus and attention to the teaching and learning process as it occurs in each classroom and school. I have seen that type of collegial, professional and effective teaching and learning environment work in my children’s schools, especially K-MS but that was in the 90s and very early 00s before the NCLB mandates. We were headed down the tracks in the right direction before NCLB and its successive iterations put us on a turntable and kicked us out heading in the wrong direction destroying the teaching and learning process/environment ever since.
The damage done by Deform, in ELA, is incalculable, and the Deformers are clueless about this.
Agreed. We need to get off the testing train that leads to the prison camps and save a generation of students behaviorism pretending to be education.
If the unions don’t make the change soon we might guess that Wexler is priming the pump for curriculum change by local government, meaning they will be dictating curriculum and segregating locally through another top down approach. That is the next rouse by the 1%’ers. The teachers will have no say in the matter. Pathways set by local government and their consultants, all who stand to profit handsomely.
We see this in our district already. Local government creating Math Packets for teachers, fresh out of college project management consultants with no background in education selling “sustainability” curriculum to District admin and school board, and school board members involved in their own inside “education/development” deals with fingers crossed that their pet projects will magically improve student outcomes.
Peter is spot on about Wexler. A closer look at local government’s ties to education reform is where to look next.
“Ultimately, her answer to the question “what to do about standardized tests” is “ignore them and teach lots of content as you know you should, while politely suggesting that test-centric accountability hawks and test manufacturers change their whole philosophy.” It would be nice to see her a bit more critical of the testing folks and a little more supportive of the teachers. Instead of rescuing the tests, let’s rescue teachers and students and education with rich content knowledge.”
I wanted to add a thank you to Mr. Greene for this wise, perceptive, important article–one of the best I’ve read in many, many years. I hope that it is read and understood by a great many decision makers. Bravo, Mr. Greene!!!
I did not mean to suggest that Mr. Greene does not, in his article, “get at the underlying issues.” He does indeed do that, and I misspoke. Importantly, he mentions BOTH the invalidity of the tests AND the insanity of wresting control from classroom teachers and putting it in the hands of top-down deformers. What I meant to say was that in a short article such as Mr. Greene’s, there isn’t space to elaborate on these–to explain why these underlying issues are determinative–why they have doomed Deform to failure. But even that is suggested by Mr. Greene’s article. I have added, above, only that we’ve been down the road of failed reform before. Altogether, Mr. Greene’s essay is an extraordinary piece of work–just brilliant. Thanks be to all the gods that someone as brilliant as he is has this national forum.
Considering the audience for which he is writing, I agree with your praise for Greene’s article(s). I think he hits the right note to reach an audience that might have bought into the “business” of education as the model for public education to follow.
Testing wastes valuable teaching time and money that would be better spent on real education. Standardized testing is useless in helping to better serve students, and it has been misused in order to promote the test and punish policies of the past twenty years. Years ago teachers had more autonomy, and students learned more as the goal was to teach rich, meaningful content. Preparing students for the future has nothing to do with performance on bubble tests.
The high-stakes standardized, summative tests are invalid, and they are pedagogically USELESS and extremely expensive, both in dollar cost and in opportunity cost.
“The high-stakes standardized, summative tests are invalid” as proven by Noel Wilson in his never refuted nor rebutted 1997 dissertation, THE most important piece of education writing in the last half century. See comment below for a summary and my comments.
Brief outline of Wilson’s “Educational Standards and the Problem of Error” found at https://epaa.asu.edu/ojs/article/viewFile/577/700 and some comments of mine. (updated 6/24/13 per Wilson email)
A description of a quality can only be partially quantified. Quantity is almost always a very small aspect of quality. It is illogical to judge/assess a whole category only by a part of the whole. The assessment is, by definition, lacking in the sense that “assessments are always of multidimensional qualities. To quantify them as unidimensional quantities (numbers or grades) is to perpetuate a fundamental logical error” (per Wilson). The teaching and learning process falls in the logical realm of aesthetics/qualities of human interactions. In attempting to quantify educational standards and standardized testing the descriptive information about said interactions is inadequate, insufficient and inferior to the point of invalidity and unacceptability.
A major epistemological mistake is that we attach, with great importance, the “score” of the student, not only onto the student but also, by extension, the teacher, school and district. Any description of a testing event is only a description of an interaction, that of the student and the testing device at a given time and place. The only correct logical thing that we can attempt to do is to describe that interaction (how accurately or not is a whole other story). That description cannot, by logical thought, be “assigned/attached” to the student as it cannot be a description of the student but the interaction. And this error is probably one of the most egregious “errors” that occur with standardized testing (and even the “grading” of students by a teacher).
Wilson identifies four “frames of reference” each with distinct assumptions (epistemological basis) about the assessment process from which the “assessor” views the interactions of the teaching and learning process: the Judge (think college professor who “knows” the students capabilities and grades them accordingly), the General Frame-think standardized testing that claims to have a “scientific” basis, the Specific Frame-think of learning by objective like computer based learning, getting a correct answer before moving on to the next screen, and the Responsive Frame-think of an apprenticeship in a trade or a medical residency program where the learner interacts with the “teacher” with constant feedback. Each category has its own sources of error and more error in the process is caused when the assessor confuses and conflates the categories.
Wilson elucidates the notion of “error”: “Error is predicated on a notion of perfection; to allocate error is to imply what is without error; to know error it is necessary to determine what is true. And what is true is determined by what we define as true, theoretically by the assumptions of our epistemology, practically by the events and non-events, the discourses and silences, the world of surfaces and their interactions and interpretations; in short, the practices that permeate the field. . . Error is the uncertainty dimension of the statement; error is the band within which chaos reigns, in which anything can happen. Error comprises all of those eventful circumstances which make the assessment statement less than perfectly precise, the measure less than perfectly accurate, the rank order less than perfectly stable, the standard and its measurement less than absolute, and the communication of its truth less than impeccable.”
In other words all the logical errors involved in the process render any conclusions invalid.
The test makers/psychometricians, through all sorts of mathematical machinations attempt to “prove” that these tests (based on standards) are valid-errorless or supposedly at least with minimal error [they aren’t]. Wilson turns the concept of validity on its head and focuses on just how invalid the machinations and the test and results are. He is an advocate for the test taker not the test maker. In doing so he identifies thirteen sources of “error”, any one of which renders the test making/giving/disseminating of results invalid. And a basic logical premise is that once something is shown to be invalid it is just that, invalid, and no amount of “fudging” by the psychometricians/test makers can alleviate that invalidity.
Having shown the invalidity, and therefore the unreliability, of the whole process Wilson concludes, rightly so, that any result/information gleaned from the process is “vain and illusory”. In other words start with an invalidity, end with an invalidity (except by sheer chance every once in a while, like a blind and anosmic squirrel who finds the occasional acorn, a result may be “true”) or to put in more mundane terms crap in-crap out.
And so what does this all mean? I’ll let Wilson have the second to last word: “So what does a test measure in our world? It measures what the person with the power to pay for the test says it measures. And the person who sets the test will name the test what the person who pays for the test wants the test to be named.”
In other words it attempts to measure “’something’ and we can specify some of the ‘errors’ in that ‘something’ but still don’t know [precisely] what the ‘something’ is.” The whole process harms many students as the social rewards for some are not available to others who “don’t make the grade (sic)” Should American public education have the function of sorting and separating students so that some may receive greater benefits than others, especially considering that the sorting and separating devices, educational standards and standardized testing, are so flawed not only in concept but in execution?
My answer is NO!!!!!
One final note with Wilson channeling Foucault and his concept of subjectivization:
“So the mark [grade/test score] becomes part of the story about yourself and with sufficient repetitions becomes true: true because those who know, those in authority, say it is true; true because the society in which you live legitimates this authority; true because your cultural habitus makes it difficult for you to perceive, conceive and integrate those aspects of your experience that contradict the story; true because in acting out your story, which now includes the mark and its meaning, the social truth that created it is confirmed; true because if your mark is high you are consistently rewarded, so that your voice becomes a voice of authority in the power-knowledge discourses that reproduce the structure that helped to produce you; true because if your mark is low your voice becomes muted and confirms your lower position in the social hierarchy; true finally because that success or failure confirms that mark that implicitly predicted the now self-evident consequences. And so the circle is complete.”
In other words students “internalize” what those “marks” (grades/test scores) mean, and since the vast majority of the students have not developed the mental skills to counteract what the “authorities” say, they accept as “natural and normal” that “story/description” of them. Although paradoxical in a sense, the “I’m an “A” student” is almost as harmful as “I’m an ‘F’ student” in hindering students becoming independent, critical and free thinkers. And having independent, critical and free thinkers is a threat to the current socio-economic structure of society.
All this general, abstract thinking about measurement and error has its place (these are fascinating topics philosophically, and thinking about them can lead to valuable insights), but I don’t think that posts about Wilson and his ideas are going to lead to people throwing out testing and grading. They are too wedded, unfortunately, to both. And, these arguments run the risk of being dismissed as fringe over-generalization. It simply isn’t the case that one can never measure whether some particular person has learned some particular, concrete thing, and the obviousness of that fact makes people dismiss such arguments. While it is completely arbitrary and subjective to claim to measure by any instrument something as vague as whether a student has “good inferencing skills” or by any standardized test whether he or she is “college and career ready,” as though every career requires the same readiness, it is not rocket science to devise methods to figure out whether you have learned that 1 squared is 1 or that Belgium is a country, not a city. I think we are better served, in combating these tests, to expose, via analysis, the stupidity of the actual questions on actual exams, the invalidity of those questions for measuring the standards that they purport to measure, the problems with the “standards” on which they are based (that’s a LONG list) AND to point to the tremendous opportunity costs of the tests–billions of dollars that could have been spent on wraparound services for poor kids and on teaching materials and facilities, billions of hours of time taken away from valuable instruction, many millions of hours taken away from lesson planning for pedagogically useless tasks like data chats and preparation of data walls and such crap, and the pushing out of valuable curricula by test prep.
Wow, Robert and not in a good sense. Where to start?
“And, these arguments run the risk of being dismissed as fringe over-generalization.”
Please explain how what Wilson says is an “over-generalization”. Now my summary may be but read his work and tell me how it is an over-generalization.
“It simply isn’t the case that one can never measure whether some particular person has learned some particular, concrete thing, and the obviousness of that fact makes people dismiss such arguments.”
When has someone measured those things that you reference, even supposed math facts. Counting the number of correct answers is not a measurement. By definition in order to measure one must have a standard by which the measurement takes place or at least that the measuring device is calibrated against said standard. Show me (I’m from Missouri, eh) that standard, how it came about and who agreed to it. Show me an exemplar of that standard and show me how the tests and all the questions are calibrated against said standard. . . . How is the measuring taking place?
You can’t, plain and simple.
To continue to use a false definition of measurement is one of the fundamental problems with the standards and testing regime.
You seem to believe that people can’t understand, that they have been so conditioned to the measurement meme that it is hopeless to even attempt to set their thinking right. I don’t agree. It seems to be an elitist position that the peons can’t understand what Wilson is saying. I can’t agree with that either.
Now the rest of your reasons against the standards and testing regime I agree with but it baffles my brain to see you write, what you have written about Wilson’s and my analysis as beyond and above the average Joe, because as I talk with Average Janes they understand about the lack of validity, the lack common sense that is the measuring meme. Not that they may come back with something to the effect of “But we have to be able to evaluate where the student is”. And, although I do disagree to some degree with that thought, I understand what they are saying.
Just don’t call it measuring.
And get rid of the false standards.
Duane, it simply is not the case that it’s always an arbitrary decision as to what is to be noodled (I won’t call it measurement because that seems to bother you). Yes, I can teach guitar without having students learn how to read music, but if they learn to read music, a lot will be open to them that wasn’t. And I can teach guitar without their learning any music theory, but again, a little music theory will open up a lot of possibilities for them–they will be able to play more, and more easily, if they understand how musical works are constructed. So, in general, I know, when teaching guitar, that it’s a good thing for my students to learn, for example, that the lines on the treble clef stand for E, G, B, and D and that the notes in a C chord are C, G, and E. And it’s a SIMPLE matter for me to noodle whether they do or not. There. I didn’t call it measuring.
I wouldn’t call it noodling either. Here is what noodling is all about:
I didn’t say that is an “arbitrary decision” to ASSESS, EVALUATE, JUDGE, etc. . . what it is that is being learned. Putting up a false attribution is not a good rhetorical tactic.
As it is you did not address my concerns. . . “read his [Wilson’s] work and tell me how it is an over-generalization.” Or “Counting the number of correct answers is not a measurement . . . How is the measuring taking place?”
As I used to tell the students when handing out tests “Have at it and have fun!”
I didn’t say that is an “arbitrary decision” to ASSESS, EVALUATE, JUDGE, etc. . . what it is that is being learned. Putting up a false attribution is not a good rhetorical tactic.
As it is you did not address my concerns. . . “read his [Wilson’s] work and tell me how it is an over-generalization.” Or “Counting the number of correct answers is not a measurement . . . How is the measuring taking place?”
As I used to tell the students when handing out tests “Have at it and have fun!”
Also you stated something to the effect that Wilson’s work is “crankish”. Please explain what parts are “crankish”. Without further clarification/discussion I certainly can’t consider your opinion to be valid.
Duane, you quote Wilson as saying, “So what does a test measure in our world? It measures what the person with the power to pay for the test says it measures.” That sounds as if he and you are saying that tests never measure something actual—something needed for some particular learning to have occurred–but can measure something that the test maker arbitrarily decides to say that it measures. You are a Spanish teacher. As you know, in Spanish or in any other language, there exists a large number of words that are rare and seldom necessary for communication and a much, much smaller number of words that are extremely common and extremely valuable for communication. Charles Ogden captured this observation in his list of words that constitute what he called “Basic English,” a list into which most other statements in English could be roughly translated. So, if your aim is to teach a student to be able to hold a conversation in Spanish, you will naturally concentrate on ensuring that the student gains a command of the smaller set of words that are most common and valuable for communication in Spanish, and it’s not difficult to determine, at least roughly, what those words are or whether a given student as learned them. A problem arises, of course, when some idiot decides that given the foregoing fact about language, ALL one needs to do is to teach students some list of common words, and he or she will then be able to communicate. “A little knowledge is a dangerous thing,” said Pope. Any approach to language instruction that simply had kids memorize the meanings of a list of basic terms would be an utter failure. And it’s this kind of utter failure that we encounter in the current Deform movement.
Yes, Wilson (and Foucault, whom Wilson references) rightly claims that in any assessment, there is a power dynamic going on. But it’s simply false that the decision about what is to be learned or measured is determined solely by a decision made by the tester. Ideally, that’s determined by external factors like usefulness for communication (or for whatever else is being taught).
The point about power dynamics in assessment is an important one. It reminds me of a story told to me by Don Gray, a great professor of English at Indiana University. Many years ago, the renowned Shakespeare scholar George Lyman Kitteridge, of Harvard, came to I.U. to give a series of lectures. Now, Kitteridge, despite his learning and fame, never received a PhD. One of the professors, Gray told me, suggested to Kitteridge that he might sit for the PhD exam at almost any school—at Indiana, for example–and would simply fly through it. Kitteridge’s answer was, “Which of you is going to examine me?” I LOVE that story. But again, the fact that assessment involves unequal power does not invalidate all assessment, though it does raise an important issue that we need to think about—the importance of using that power responsibly.
My concern with this stuff is the following, Duane: There are specific reasons why the current tests are invalid, and these have nothing to do with the invalidity of all measurement, and making the claim that all measurement is invalid because there can never be, in any educational situation, ANY standard, is simply crankish, crackpot, and lends aid to the enemy—to the Deformers who want to paint all those who oppose their numerology are crackpots. Physicists constantly get letters and tomes from nonscientists who claim to have invented perpetual motion machines or to have proved that Newton or Einstein got it all wrong, and these letters and tomes have some recurring themes: The crackpot is being ignored. If the physicist would only plow through the 800 pages of the crackpot’s dissertation on Quantum Akashic Gravity, he or she would see why Newton and Einstein were wrong. Do this, and find ONE STATEMENT in the 800 pages that is incorrect. And so on.
Yes, we measure physical properties using various instruments, various physical measuring devices that serve as standards (e.g., One second is the time that elapses during 9.192631770 x 10 9 cycles of the radiation produced by the transition between two levels of the cesium 133 atom as measured by what’s known as a Cesium Fountain Clock). But one cannot conclude from this that because, in education, we rarely use physical measuring devices, no reliable or valid measurement can EVER take place. To go back to my guitar teaching example, we can take as a pretty reliable standard of what an guitarist needs to know about reading music and about music theory what is known by guitarists who play very well from music, using their knowledge of music theory. Making the claim that no measurement is ever possible in education is simply crazy. Learning one’s times table makes doing math easier. It’s a fairly simple matter to measure, using a test, whether a given student does or does not know his or her times table, and the standard is provided by an existent one can point to—by anyone else who DOES know the times table. This is not news to most anyone. So, making the claim that no measurement (or whatever else you want to call it) can ever take place in education sounds like Quantum Akashic Gravity—it just sounds kooky. And it distracts from discussion of the real, serious validity and usefulness issues with the high-stakes, standardized, summative tests, as well as from the problems with the opportunity costs of these tests and their distortions of pedagogy and curricula.
Bob, I think Duane takes issue with the fact that education uses the term measurement loosely. When I decide that a student is “proficient ” on his/her times table, I decide the standard that determines that. Someone else may choose a different one. There is no universal standard that can be reliably applied. We are not measuring a physical quality. However, it is a mistake to apply those same understanding of standards to what is meant by measurement in the social sciences. Duane argues , I think, that the term measurement should only be used when discussing physical phenomena. I’m not sure why the physical scientist should get a monopoly on the use of the term since a vast majority of words, it seems to me, have multiple meanings that are dependent on context. I do, however, agree with being cautious when using the term because it is much too easy for us to rank and sort people on the basis of such mutable scales. It is more valuable to talk about evaluating, assessing, or critiquing performance based on expert knowledge/professional opinion. How many of us at some point in our lives did not feel like we were incapable of performing at some level because of some test told us so? We have seldom made a point of adding all the caveats that should be attached to any such ranking in a way that is meaningful to and supportive of an individual’s understanding of their own performance.
I emphatically agree with “being cautious when using the term”!!! There are things that are easily measured. There are a LOT that that cannot be. And it is extremely important to recognize the difference–in particular, to understand that the further we get from concreteness, the less measurement can have any validity. And there is a terrible irony at the heart of the Ed Deform mantra that “you get what you measure.” The high-stakes standardized tests in ELA ask students to “apply” standards [sic] from the Gates/Coleman bullet list to trivial exercises related to random snippets of text. And that is what we get–triviality. The tests don’t measure what they purport to measure–the blitheringly general and abstract “skills” on the Gates/Coleman list. What they do measure–the ability to answer such exercises in the ways that the test makers envision–is completely divorced from real reading and response.
And so, it is purest numerology to attempt to measure something as vague as “the student’s ability to make inferences from texts”–the notion that one can reliably and validly do THAT is simply crazy. Something can be accurately measured only if it can be made concrete enough to be accurately (and repeatedly) measured. When we substitute for the general thing that we wish to measure some particular, concrete, objective measurement–we are measuring the later, not the former. This is the idea that lies behind the old saw that IQ is not intelligence but, rather, whatever it is that intelligence tests measure. So, I have no disagreement there.
“I think we are better served, in combating these tests, to expose, via analysis, the stupidity of the actual questions on actual exams…”
Good luck with that. Back in the day when we were able to see questions, fellow teachers found questions that were somewhere between silly and absurd. they were treated to a bureaucratic stream of excrement worthy of any epithet.
you may be correct that calling attention to the illogic of testing will not sway the minds of the average taxpayer, whose chief concern is the bills in his pocket instead of the ones in the legislative hopper, but logic has no rival when declaring to the world that this is a nude emperor.
The current standardized tests are demonstrably invalid. They are demonstrably pedagogically useless. And they demonstrably burden the system with enormous opportunity costs. It seems to me that we should concentrate on those matters rather than getting off onto an admittedly fascinating but less immediately productive and applicable discussion of the epistemology of error and measurement. All that seems to me like holding discussions of the aesthetics of hull shapes when there is a hole in the hull. Don’t get me wrong–I’m a huge fan of epistemology, but in its place.
And, I must say, I’ve made several passes at the book by Wilson referenced above, and while it has interesting moments, a lot of it seems extraordinarily crankish. https://en.wikipedia.org/wiki/List_of_topics_characterized_as_pseudoscience
Greene’s articles in Forbes are outstandingly well written and of yuge importance. So is debating on this blog, especially when everyone involved is right. The billionaires have the money; we have the numbers. We have to use our numbers to overcome their money. That means:
(A) presenting scientific facts to battle their “think” tank propaganda
(B) grassroots organizing to continue and build the OptOut movement, including exposing every nonsense question about talking pineapples we can find
(C) organizing teachers unions to protect teachers and students instead of political connections, which means refusing to support bipartisan centrists who allow Wall Street to hold all the strings of power
(D) teaching and researching
(E) all of the above
You know the answer. It’s E. This hearty debate here in Diane’s virtual living room occurs with perfect timing for me, as tonight I get to do one of the best things I can to fight the testing industry’s lobbying grip on the federal government. Bernie’s in town! I’m going to his rally. We’ve already helped him get Elizabeth Warren to call for an end to testing. Others will follow. We have to use our intellects and knowledge, but we also must use our hearts. I feel outrage against testing. I feel it. I feel the Bern. I’m going to get LOUD tonight! That’s what we need to do, all of the above. Continuously. Loudly.
Thank you, LCT.
Stay strong!
“The billionaires have the money; we have the numbers. We have to use our numbers to overcome their money.”
As much as I want to believe you and agree, in theory, with your answers A through E, LCT, I think it is wishful thinking in the real world. In my community, I would assume that am the only parent/citizen with no professional stake in education who understands and agrees with you. In fact, I think many of the teachers in the district my children are in would disagree with you, perhaps not in public, but certainly in private and with their votes. And I think that is just human nature. Education does not rise to become an important issue for those who aren’t—or think they aren’t—a part of the education community. It is an issue that is ignored. The only thing that motivates some parents is school sports and supporting other extracurricular activities. Not one of these “engaged” parents cares one iota about the issues that are discussed in this forum.
We do not have the numbers. We don’t even have a coherent narrative that breaks through the little garbage that most focus on. Here in Ohio that is proven every day. We elect the enemies of education and promote them. Our neighbors (very) occsionally get shrill about the evils of ECOT and still don’t make the connection with the K12 marketing blizzard of the Ohio Virtual Academy. When I tell people it’s the same wolf in scholar’s clothing, I get quizzical looks.
We have administrators who buy into the status quo because it creates job security. And we have absolutely no pushback from parents, teachers or citizens. We do not have the numbers.
A Deformer group recently issued a purported survey suggesting that parents support standardized testing. I have rarely, in my time as a teacher or as an educational publishing executive, met one who does. I’m pretty certain that there is overwhelming opposition to standardized tests among parents, with a few exceptions–wealthy parents whose kids do well on them. I’m pretty sure that an unbiased survey would show this. Peter Greene did an article about that biased Ed Deform survey. It was, as usual, great reading.
Greg, if education issues are ever put to a popular vote, you will be surprised to learn that parents care and are better informed than you know. The deformers will go to great lengths to keep their agenda off the ballot. It will lose and you will discover that we do have the numbers.
Thank YOU, Diane. Keep fighting.
Peter ends with a great last line: “Instead of rescuing the tests, let’s rescue teachers and students and education with rich content knowledge.”
Yes, sir!
Amen to that!!!!
We need extensive organization and support, otherwise the change will occur on a glacial timescale, maybe even evolutionary, if at all.
That we do. The problem is that the Ed Deformers are extremely well funded, and in both state capitals in Washington, money talks. So, unless the unions get behind the anti-standardized-testing movement, we are in for a lot more of the same. I’ve devoted a lifetime to teaching ELA and to writing and editing ELA textbooks. I’ve lived to see, as a result of these state-test-based Deforms, the replacement of instruction in literature and writing, in textbooks and in depersonalized education software, by exercises modeled on state test questions in which students “apply” standards [sic] from the puerile Gates/Coleman bullet list to random snippets of text. This trivialization of ELA has been disastrous. The most brilliant and talented ELA teachers and instructional materials creators I know are all jumping ship because they just can’t stand it anymore. A whole generation of students has had humane education in literature and writing stolen from them. What’s happening is a nightmare, and the Deformers are clueless about the damage they’ve done. And this won’t end until the federal testing mandate ends.
Too many unions are so much political machinery and there is a lot of distrust of unions, even from within.
I think the organization has to be bigger than unions. But I don’t know what we can expect from this country, given . . . It has likely been this way for many decades, it’s just been clearly exposed now.
Still, it’s a specific web of people we need convinced and organized in some way.
Two words: National curriculum.
If you listen closely you can hear James Madison and Thomas Jefferson’s eyes rolling.
But one has to admire BA’s brevity to succinctly expose his/her stupidity. Did it in two words.
Aie yie yie. Yeah, that’s a great solution. Let’s create another system of national Thought Police to micromanage education. #&#&#^@^!#!!!!
Wait–am I crazy or are these Peter Greene pieces from 2018?!
It’s probably too late to be read (because, Duane, you are way up there at 2:59 PM what is now yesterday, 7/26, but I always agree w/Swacker as to what he says about “standardized” testing. It is looooong past the time of “discussion.” WHY are we
still TALKING about this?!
As someone who worked very hard in the opt out movement (standing in front of schools, passing out opt-out literature to parents, organizing meetings, etc.), it would appear that it is, indeed, up to parents (& retired teachers are ready & waiting to help you!) to STOP.THIS.NONSENSE.NOW. THIS YEAR.
No more ed. money to Pear$$$on. No more test preps in lieu of civics classes, no more closing “failing” schools (based on failing tests–tests that not only do not measure ANYTHING & give us NO informations, but te$ts that are pure & utter NONSENSE. Te$t$ that are unregulated & te$ting companie$ that are NOT held accountable.
What did Leonie Haimson & parents in NYC do when the “Pineapple Question” was revealed? Why, they marched on Pear$on headquarters–with their children–dressed as pineapples. What does it take to get people marching in the streets until the State Stuperintendents & Big, Bad Boards STOP the yearly (& I mean ALL year–a total year wa$ted on te$t prepping–actually, year$ & year$ & year$.) theft of real education from our children?
How is it that Puerto Rican citizens were victorious in gaining the resignation of their governor? How did the Chicago Teachers Union–&, later, some other state unions–gain contracts?
They TOOK TO THE STREETS.
Go protest in front of your Pear$on$ (& any other te$ting companie$ that are left; Pear$on does have a monopoly).
No more discussion. ACTION.
And this is the last time I am taking part in a “discussion” on thi$ $ubject, becau$e it’$ all about the Benjamin$, the clo$ing of $chool$/di$enfranchi$ement of real educators,
& the withholding of an education to “other people’s children.”
With the latter, Walmart will get their (below) minimum wage, no benefits workers, Amazon will run their modern-day sweatshops (until all their employees are replaced w/robots), & poverty, drug addiction & early deaths will continue to swell, just as the climate crisis.
(An aside–did you all know that the Tour de France was halted because there were- 1) mudslides, then, even more strangely–2) blizzards–?)