Bob Shepherd, a frequent commenter here, has been a curriculum writer, as assessment developer, a publisher, and a classroom tea her. As frequent readers of this log know, he is also a polymath, with a broad, nearly encyclopedic range of knowledge.
In this essential post, he explains why standardized testing is invalid and useless for accountability purposes.
They do not measure what they claim to measure.
Here is a brief excerpt from a brilliant explanation:
Nothing that students do on these exams even remotely resembles what real readers and writers do with real texts in the real world. Ipso facto, the tests cannot be valid tests of actual reading and writing. People read for one of two reasons—to find out what an author thinks or knows about a subject or to have an interesting, engaging, significant vicarious experience. The tests, and the curricula based on them, don’t help students to do either. Imagine, for example, that you wish to respond to this post, but instead of agreeing or disagreeing with what I’ve said and explaining why, you are limited to explaining how my use of figurative language (the tests are a miasma) affected the tone and mood of my post. See what I mean? But that’s precisely the kind of thing that the writing prompts on the Common [sic] Core [sic] ELA tests do and the kind of thing that one finds, now, in ELA courseware. This whole testing enterprise has trivialized responding to texts and therefore education in the English language arts generally. The modeling of curricula on the all-important tests has replaced normal interaction with texts with such freakish, contorted, scholastic fiddle faddle. English teachers should long ago have called BS on this.
Open the link and read it all.
Thank you, Bob Shepherd.
Excellent piece.
Sure hope the new Sec. of Education understands ALL the points made. Duncan sure didn’t and neither did Obama.
As far as I’m concerned, Bob has the first and last word on this subject.
Have you read Noel Wilson’s “Educational Standards and the Problem of Error”? His is THE first and last word (Sorry, Bob, but I call it as I see it.)If not please read it to understand all the onto-epistemological errors, falsehoods and wrong assumptions that render the usage of results of standardized tests completely invalid. See: https://epaa.asu.edu/ojs/article/viewFile/577/700
Brief outline of Wilson’s “Educational Standards and the Problem of Error” and some comments of mine. (updated 6/24/13 per Wilson email)
1. A description of a quality can only be partially quantified. Quantity is almost always a very small aspect of quality. It is illogical to judge/assess a whole category only by a part of the whole. The assessment is, by definition, lacking in the sense that “assessments are always of multidimensional qualities. To quantify them as unidimensional quantities (numbers or grades) is to perpetuate a fundamental logical error” (per Wilson). The teaching and learning process falls in the logical realm of aesthetics/qualities of human interactions. In attempting to quantify educational standards and standardized testing the descriptive information about said interactions is inadequate, insufficient and inferior to the point of invalidity and unacceptability.
A major epistemological mistake is that we attach, with great importance, the “score” of the student, not only onto the student but also, by extension, the teacher, school and district. Any description of a testing event is only a description of an interaction, that of the student and the testing device at a given time and place. The only correct logical thing that we can attempt to do is to describe that interaction (how accurately or not is a whole other story). That description cannot, by logical thought, be “assigned/attached” to the student as it cannot be a description of the student but the interaction. And this error is probably one of the most egregious “errors” that occur with standardized testing (and even the “grading” of students by a teacher).
Wilson identifies four “frames of reference” each with distinct assumptions (epistemological basis) about the assessment process from which the “assessor” views the interactions of the teaching and learning process: the Judge (think college professor who “knows” the students capabilities and grades them accordingly), the General Frame-think standardized testing that claims to have a “scientific” basis, the Specific Frame-think of learning by objective like computer based learning, getting a correct answer before moving on to the next screen, and the Responsive Frame-think of an apprenticeship in a trade or a medical residency program where the learner interacts with the “teacher” with constant feedback. Each category has its own sources of error and more error in the process is caused when the assessor confuses and conflates the categories.
Wilson elucidates the notion of “error”: “Error is predicated on a notion of perfection; to allocate error is to imply what is without error; to know error it is necessary to determine what is true. And what is true is determined by what we define as true, theoretically by the assumptions of our epistemology, practically by the events and non-events, the discourses and silences, the world of surfaces and their interactions and interpretations; in short, the practices that permeate the field. . . Error is the uncertainty dimension of the statement; error is the band within which chaos reigns, in which anything can happen. Error comprises all of those eventful circumstances which make the assessment statement less than perfectly precise, the measure less than perfectly accurate, the rank order less than perfectly stable, the standard and its measurement less than absolute, and the communication of its truth less than impeccable.”
In other words all the logical errors involved in the process render any conclusions invalid.
The test makers/psychometricians, through all sorts of mathematical machinations attempt to “prove” that these tests (based on standards) are valid-errorless or supposedly at least with minimal error [they aren’t]. Wilson turns the concept of validity on its head and focuses on just how invalid the machinations and the test and results are. He is an advocate for the test taker not the test maker. In doing so he identifies thirteen sources of “error”, any one of which renders the test making/giving/disseminating of results invalid. And a basic logical premise is that once something is shown to be invalid it is just that, invalid, and no amount of “fudging” by the psychometricians/test makers can alleviate that invalidity.
Having shown the invalidity, and therefore the unreliability, of the whole process Wilson concludes, rightly so, that any result/information gleaned from the process is “vain and illusory”. In other words start with an invalidity, end with an invalidity (except by sheer chance every once in a while, like a blind and anosmic squirrel who finds the occasional acorn, a result may be “true”) or to put in more mundane terms crap in-crap out.
And so what does this all mean? I’ll let Wilson have the second to last word: “So what does a test measure in our world? It measures what the person with the power to pay for the test says it measures. And the person who sets the test will name the test what the person who pays for the test wants the test to be named.”
In other words it attempts to measure “’something’ and we can specify some of the ‘errors’ in that ‘something’ but still don’t know [precisely] what the ‘something’ is.” The whole process harms many students as the social rewards for some are not available to others who “don’t make the grade (sic)” Should American public education have the function of sorting and separating students so that some may receive greater benefits than others, especially considering that the sorting and separating devices, educational standards and standardized testing, are so flawed not only in concept but in execution?
My answer is NO!!!!!
One final note with Wilson channeling Foucault and his concept of subjectivization:
“So the mark [grade/test score] becomes part of the story about yourself and with sufficient repetitions becomes true: true because those who know, those in authority, say it is true; true because the society in which you live legitimates this authority; true because your cultural habitus makes it difficult for you to perceive, conceive and integrate those aspects of your experience that contradict the story; true because in acting out your story, which now includes the mark and its meaning, the social truth that created it is confirmed; true because if your mark is high you are consistently rewarded, so that your voice becomes a voice of authority in the power-knowledge discourses that reproduce the structure that helped to produce you; true because if your mark is low your voice becomes muted and confirms your lower position in the social hierarchy; true finally because that success or failure confirms that mark that implicitly predicted the now self-evident consequences. And so the circle is complete.”
In other words students “internalize” what those “marks” (grades/test scores) mean, and since the vast majority of the students have not developed the mental skills to counteract what the “authorities” say, they accept as “natural and normal” that “story/description” of them. Although paradoxical in a sense, the “I’m an “A” student” is almost as harmful as “I’m an ‘F’ student” in hindering students becoming independent, critical and free thinkers. And having independent, critical and free thinkers is a threat to the current socio-economic structure of society.
Thanks, Duane: Found it, bookmarked it, and I will read it as time permits….
Thanks, Senor. Short, sweet & need to repeat.
Duane, I would correct you slightly:
Noel Wilson may have the first word.
But you surely will have the last! (As long as you are breathing, at least)
The error that the “education as a business” proponents make is in assuming that “standardizing” the process necessarily leads to higher quality output.
For a manufacturing process, that is actually true: the greater the standardization and control over the process, the lower the defect rate.
But, of course, the error is in assuming that an idea developed for manufacturing applies to education, where the goal is entirely different.
In manufacturing, the goal is to eliminate as much variation as possible and the best way of doing that is to carefully control the process.
With education, the goal should certainly not be elimination of variation, since that is the source of all creative thinking, which is more important today than it ever had been.
Yong Zhao has pointed this out many times, but of course, the billionaires, politicians and other widget manufacturers never listen to people who actually know what they are talking about.
There is an irony involved in this case.
Even in business, it is the “outliers” (Elon Musk, Steve Jobs and others) who drive change with new ideas.
It is only after the fact — after a new idea for a product like a smartphone or electric car has been proposed — that the mindless manufacturing focus on reducing defects goes into effect .
Evolution works this way as well. Random mutations sometimes produce characteristics that make an organism better adapted to its environment and it is only after the initiall change that the process settles down into a new stable state.
If all variation were rejected without regard for whether it makes an organism better adapted, there would be no evolution and hence no development of higher life forms.
The reason that standardization works well for manufacturing is that manufactured parts do only one thing and often have to fit together with other parts. For such a case , it is advantageous to eliminate as much variation as possible.
But what should be obvious is that humans don’t just do one thing, so the idea that one should apply a manufacturing model to education is just ridiculous.
Seems to me that many who push standardization in the teaching and learning process have never been involved in the manufacturing of parts/end product. They don’t know how to distinguish what you say in your posts, assuming that if it is standardized it has to be better.
Those of us who have been in those manufacturing environments seem to understand better than those who haven’t been there.
Duane
You may be right that I am giving the deformers too much credit by assuming they know something about manufacturing.
I gained my knowledge of manufacturing from a couple years spent on the factory floor of a company that made bows for hunting and Olympic archery. I did a lot of different stuff while I was there but a lot of it was related to quality control.
I’ve been a screw machine operator and also was a production scheduler/materials manager for a metal building manufacturer. Good quality data/information is vital to the day to day operations. Standardized test data comes nowhere near being “good quality” and is in fact, invalid. Crap in nets crap out and that can’t be in manufacturing. And it shouldn’t be acceptable in the teaching and learning process.
OK, Duane. Step outside and say that. LMAO.
I still don’t get this no intellectual attainments can be measured stuff. Clearly, obviously, SOME can, and fairly easily. Suppose that you are a student of Japanese. I can give you a test and find out with a great deal of certainty (and with repeatability across administrations and to differing students) whether you know your kana and how many kanji you can recognize. If you are an elementary school student of math, I can give you a test and find out with a great deal of certainty how much of the times table through 12 x 12 you have committed to memory. People run into issues when they try to measure things that are not measurable or not simply and accurately measurable or for which there exist no defensible external benchmark against which to measure it.
With that caveat, a lot of the stuff that you say above is true.
Okay, I’m outside!
One is not measuring what the student has learned. One is assessing, evaluating, judging, appraise, roughly gauge, etc. . . . but not measure. Not all assessments, evaluations, judgments, appraisals etc. . . are a form of measuring.
It is precisely the misuse of the term measure which lends that false scientific sheen of supposed validity and reliability (where there truly isn’t any) and objectivity when there isn’t any.
I have no problem with true measurement, one might consider the ability to accurately measure and weigh to be the foundation of supposedly civilized mankind.
OK. I am, of course, entirely sympathetic to this POV because you lay your finger on the problem with the Ed Deform agenda. These people think themselves able to assess or measure or whatever you want to call it more, and more precisely, than they can. But this metaphorical extension of terms used to describe physical characteristics to mental ones is, of course, ubiquitous. Emerson, in his essay entitled “Language,” gives a lot of examples: right, from not stepping off a straight (or right) path; inspiration, from breathing in; and so on. The question becomes, well, how good is that metaphor? Well, in physical measurement, we have a standard against which the object under consideration is compared, like the old French prototypical meter bar. And in the case of measurement of a person’s knowledge of kanji, we can invoke the knowledge of kanji of people who actually have that knowledge, which means that the comparison holds up and is justified. Amusingly, the Ed Deformers like to think themselves very scientific but never actually do CLOSE READING, or observation and analysis, of the scam tests on which their numerology is based.
Duane, brother, keep hammering at this testing nonsense.
My response is to the points made in the entry on measurement below.
How’s this for an analogy, Bob? Duane Swacker : GregB :: Noel Wilson : Willy Brandt
This is one of the most comprehensive analysis on the inadequacy of high stakes ELA standardized testing. Data collection, when politically motivated, does not deliver results that are free from bias. In fact, there is a whole segment of research dedicated to the bias of all reductionist standardized testing. So-called reformers have used the results of standardized testing to punish teachers, schools and students. This has resulted in closing public schools and sometimes transferring students to private schools of questionable quality. This process has failed to deliver improved outcomes for poor students. Standardized testing has been used to justify the privatization of public education and the placement of mostly poor minority students in separate and unequal schools.
If only the PARENTS would understand! The false scores created by these tests drive parents into a competition frenzy. Parents refuse to believe the tests are a sham as long as the fake data “proves” that their children are better than “the others”( Zillow!). I’m sorry! I know this is a blog about the merits of a decent public education, but I am so happy that I pulled child #2 out of our test centric/common cored public school system and put him into a private school that values the whole child (think public education of the 1970’s-mid 80’s) and not the data from standardized tests and test prep. Public schooling has become inhumane in it’s main purpose and the kids are suffering as a result (in school and at home from their crazy parents)….just look at the rise in suicide rates and other mental health issues in the adolescent population over the last 10-15 years.
When working people are scrambling around competing with and blaming each other, they cannot see that it is the “invisible hand” that is choking them.
Maybe? But I got tired of talking to other parents and admin about how bad this has gotten. I got tired of fighting against the standardized testing regime in my state (MD…no Opt out allowed/I had to REFUSE and jump through hoops) My own sister left the profession with over 30 yrs of teaching experience and has nothing good to say about what public education has become over the last 15 yrs. Test prep as curriculum is not an education. Not allowing children to express how they think or feel is not an education. Pushing kids to perform like circus animals for data is not an education. This is insanity….but the parents love the numbers and they just don’t care.
a perfect turn of phrase: “Pushing kids to perform like circus animals for data is not an education.”
“In this essential post, he explains why standardized testing is invalid and useless for accountability purposes. They do not measure what they claim to measure.”
Again, Noel Wilson explains in his shorter read “A Little Less than Valid: An Essay Review. An excerpt:
To the extent that these categorisations are accurate or valid at an individual level, these decisions may be both ethically acceptable to the decision makers, and rationally and emotionally acceptable to the test takers and their advocates. They accept the judgments of their society regarding their mental or emotional capabilities. But to the extent that such categorisations are invalid, they must be deemed unacceptable to all concerned.
Further, to the extent that this invalidity is hidden or denied, they are all involved in a culture of symbolic violence. This is violence related to the meaning of the categorisation event where, firstly, the real source of violation, the state or educational institution that controls the meanings of the categorisations, are disguised, and the authority appears to come from another source, in this case from professional opinion backed by scientific research. If you do not believe this, then consider that no matter how high the status of an educator, his voice is unheard unless he belongs to the relevant institution.
And finally a symbolically violent event is one in which what is manifestly unjust is asserted to be fair and just. In the case of testing, where massive errors and thus miscategorisations are suppressed, scores and categorisations are given with no hint of their large invalidity components. It is significant that in the chapter on Rights and responsibilities of test users, considerable attention is given to the responsibility of the test taker not to cheat. Fair enough. But where is the balancing responsibility of the test user not to cheat, not to pretend that a test event has accuracy vastly exceeding technical or social reality? Indeed where is the indication to the test taker of any inaccuracy at all, except possibly arithmetic additions?
A Little Less than Valid: An Essay Review
http://edrev.asu.edu/index.php/ER/article/view/1372/43
Continuing in response to “They do not measure what they claim to measure.”
They do not measure anything at all. (one of the many onto-epistemological assertions that is false on its face)
The most misleading concept/term in education is “measuring student achievement” or “measuring student learning”. The concept has been misleading educators into deluding themselves that the teaching and learning process can be analyzed/assessed using “scientific” methods which are actually pseudo-scientific at best and at worst a complete bastardization of rationo-logical thinking and language usage.
There never has been and never will be any “measuring” of the teaching and learning process and what each individual student learns in their schooling. There is and always has been assessing, evaluating, judging of what students learn but never a true “measuring” of it.
But, but, but, you’re trying to tell me that the supposedly august and venerable APA, AERA and/or the NCME have been wrong for more than the last 50 years, disseminating falsehoods and chimeras??
Who are you to question the authorities in testing???
Yes, they have been wrong and I (and many others, Wilson, Hoffman etc. . . ) question those authorities and challenge them (or any of you other advocates of the malpractices that are standards and testing) to answer to the following onto-epistemological analysis:
The TESTS MEASURE NOTHING, quite literally when you realize what is actually happening with them. Richard Phelps, a staunch standardized test proponent (he has written at least two books defending the standardized testing malpractices) in the introduction to “Correcting Fallacies About Educational and Psychological Testing” unwittingly lets the cat out of the bag with this statement:
“Physical tests, such as those conducted by engineers, can be standardized, of course [why of course of course], but in this volume , we focus on the measurement of latent (i.e., nonobservable) mental, and not physical, traits.” [my addition]
Notice how he is trying to assert by proximity that educational standardized testing and the testing done by engineers are basically the same, in other words a “truly scientific endeavor”. The same by proximity is not a good rhetorical/debating technique.
Since there is no agreement on a standard unit of learning, there is no exemplar of that standard unit and there is no measuring device calibrated against said non-existent standard unit, how is it possible to “measure the nonobservable”?
THE TESTS MEASURE NOTHING for how is it possible to “measure” the nonobservable with a non-existing measuring device that is not calibrated against a non-existing standard unit of learning?????
PURE LOGICAL INSANITY!
The basic fallacy of this is the confusing and conflating metrological (metrology is the scientific study of measurement) measuring and measuring that connotes assessing, evaluating and judging. The two meanings are not the same and confusing and conflating them is a very easy way to make it appear that standards and standardized testing are “scientific endeavors”-objective and not subjective like assessing, evaluating and judging.
That supposedly objective results are used to justify discrimination against many students for their life circumstances and inherent intellectual traits.
C’mon test supporters, have at the analysis, poke holes in it, tell me where I’m wrong!
I’m expecting that I’ll still be hearing the crickets and cicadas of tinnitus instead of reading any rebuttal or refutation.
“…where is the balancing responsibility of the test user not to cheat..”
Where indeed? Is this not the real question of our time? Where does the ruling class of people in society confront the standards of behavior required of them by virtue of the position of their ruling? Surely it is in the realm of what we call fairness.
Corruption is one of the central themes of our time. Honesty is getting hard to come by.
Bingo. The result of trickle-up economics. As of 1 yr ago, the top 1% held roughly the same amount of assets as those in the 50th to 90th %ile combined– much of the middle and upper-middle classes. Policy is inevitably skewed entirely now not to what works for the public, nor does it concern itself with superannuated concepts such as fairness and honesty. It’s just all about what works to preserve and expand the assets of the 1%.
I beg to differ. This is how I wrote about it in this piece https://fordhaminstitute.org/national/commentary/play-doh-plato-all-students-need-grapple-grade-level-text
Here’s an excerpt:
I am convinced that below-level readers should spend most of the time reading on-level text—with support. In California, students take the CAASP beginning in the third grade. Last year, I attended a two-day workshop on this assessment prior to teaching a third grade class once a week. Up until then, I had spent the majority of my time at the elementary level working with small groups of struggling first and second graders, with students who could not read the words and therefore had no gateway to comprehension.
This workshop was an eye-opener, particularly the deep dive into the performance task, whereby students analyze multiple articles on one topic and then use evidence from these articles to synthesize a response to a given “task”—producing either narrative, informative, or argumentative writing. I quickly realized that this performance task was an authentic assessment, and acquiring these skills in elementary school would certainly make students middle and high school “ready.” But it was also obvious that without explicit and systematic instruction in how to analyze text and synthesize a response, the majority of my third graders, especially my below-level readers, would not stand a chance at achieving proficiency with this “performance.”
You mention analysis and synthesis in your response. These are generally considered ‘higher order thinking skills.’ If teachers really want to understand what students know, formative assessments provide a lot more useful information about how much students understand than standardized tests.
In ELA subject matter, teachers can find out much more about what students know and understand through a written response that generally sheds light on what a student understands. Written responses demonstrate how well a student has a command of mechanics as well as his or her ability to draw conclusions, analyze and synthesize real content. These types of assessments are examples of performance based assessment.
Years ago teachers were working on portfolio assessments. When NCLB was adopted, students, teachers and school policies were forced to adopt meaningless reductionist, standardized testing. Teachers been rats in a maze ever since billionaires decided to destroy public education. Standardized tests are “weapons of math destruction.”
I don’t know where you taught, but in my district “academic freedom” prevents mandating classroom assessments. And even if it were possible, a mandate doesn’t guarantee compliance. As a parent, I would not want to depend on the serendipity of this subjective system of accountability.
Some parts of a formative assessment may be discreet, as Bob shows in his post. For example, one question may be to require a student to write a metaphor. The performance of the student determines if the student understood the concept.
I am old enough to remember Blue Books in college. I had to write written responses for most of my humanities and social science courses. I had to defend my statements, provide examples, analyze and synthesize material. The responses had to meet the requirements of the question in order to get full credit. These tasks were not really subjective. They combined thinking and writing. Responses had to meet certain clear criteria.
cx: teachers have been rats in a maze
Note that the classroom assessments disparaged above (grades given by high-school teachers) have always been far better predictors of success in college than are scores on a standardized test like the SAT or ACT.
Bob,
High school grades are not the best single predictor of college success at UC schools. See https://senate.universityofcalifornia.edu/_files/underreview/sttf-report.pdf
Quoting from the faculty report: “The STTF found that standardized test scores aid in predicting important aspects of student success, including undergraduate grade point average (UGPA), retention, and completion. At UC, test scores are currently better predictors of first-year GPA than high school grade point average (HSGPA), and about as good at predicting first-year retention, UGPA, and graduation.”
In any case, I advocate using multiple measures of academic potential rather than any single measure for admission. More information is better than less.
People repeat the things they hear that fit their conclusions.
Harriet Janetos, I am puzzled by your post. I read and re-read your article. Where does taking annual stdzd state tests fit in? Your article is right on, IMHO. Yes, absolutely: help struggling students read grade-level texts: “reducing content is not the direction we want to be headed.” Do it through scaffolding, instructional support, well-chosen reading strategies, intensive small-group work on decoding multisyllabic words, etc.
I went online and found released examples of SBAC Performance Tasks and supporting material. These are great little units for classroom reading/ writing projects, ready to go. What value is added by having students “perform” them on a timed basis? [I would venture to say if they have to do that at a keyboard, that might actually subtract value]. It is discouraging that you think grading the results yourself is “serendipitous” and “subjective.” You have the rubric in front of you, and a lot more training and experience in the subject than any SBAC-contracted scorer.
Teaching Economist: https://edpolicyinca.org/publications/predicting-college-success-how-do-different-high-school-assessments-measure-2019
And this is not the only study to have found this to e true–that high-school grades are better predictors of college success than is the SAT. There have been many. I worked on K–12 English textbooks for years and often had to edit or write SAT prep stuff, so I’ve long had an interest in this subject. Over the years, I’ve seen tons of studies of this topic.
Teaching Economist: There is a LONG history of study of this subject of the relative merits of the SAT and high-school grades as predictors of college success that, it seems, you aren’t familiar with. The SAT used to be called The Scholastic Aptitude Test because it was billed as a predictor of college success. However, there was so much research showing that it wasn’t a particularly good predictor–a particularly good measure of Aptitude–that under pressure, over many years, the College Board finally changed the name to The Scholastic Achievement Test. However, it wasn’t a particularly good measure of achievement, either (among other things, it didn’t cover enough ground; high school achievement includes things like knowledge of civics and American history not covered by the exam). So, they finally just decided to call the thing the SAT.
What the long history of the SAT amounts to is a very lucrative product, originating out of the eugenics movement, continually looking, in vain, for a justification of its existence.
Being a white, middle-class boy who went to an extremely well-funded public school, I personally benefited from the SAT (and the GRE). I would have no issue with this thing if it existed so that students who had screwed around in high-school and later got serious about their educations could have an alternative means of showing, to some extent, that they could handle first-semester college work. But as a general predictor of college success, the SAT never was all that good, especially beyond the first year.
You partook of the wrong colored Kool Ade! Really!….. Fordham Institute? Anything coming out of that Stink Tank reeks of disruption and deform.
Thank you for your objective analysis of my piece. I will take your comments into serious consideration as I strive to close the achievement gap for my children of color. Gotta go–distance teaching awaits!
Harriett Janetos,
The concerning thing is that you are a reading specialist who seems to be admitting that you have no idea how your students are doing without having them take a standardized test, and the results of the standardized test your students take is the ONLY way that you have any idea of their reading competency, and it is that score on the standardized test that you are absolutely positive is the perfect reflection of your students’ abilities.
Do you tell the parents of the kids that you teach that you don’t have a clue about their reading abilities without having their score on a timed standardized exam?
Somehow I doubt that the parents of privileged children in private schools hear that from their teachers — their teachers saying that the sum total of their child’s ability is what teachers like you learn from their standardized test score because you have no ability to know without that score.
The reason there is such a backlash against standardized tests is that middle class college educated parents in public schools have rebelled and aren’t buying into them. The fact that you are FORCING poor kids to be judged in a way that privileged children are not speaks a lot about your own prejudices and biases.
A good way for reading and/or classroom teachers to gather useful information of students strengths and weaknesses is to give students an IRI, an informal reading inventory. An IRI can help teachers to plan for instruction. A percentile on a standardized test will yield a rough grade level score, but it is not specific enough to help teachers plan. It also does not help when the scores arrive in June. If teachers know what they are doing, they can make their own IRI, but I see publishers are now selling IRI kits.
Are you suggesting that weighing the cow helps her gain weight? Most cows gain weight when they eat, not when they weigh.
Of course you witnessed extraordinary things in the workshop. You saw students engaged in learning something, a phenomenon that has led teachers into the classroom for many years. Something is leading them away now. That thing is the incessant testing of students and the constant hassle from administrations which value these false numbers over experiences.
If we continue the current test and punish attitude, there will be no more teachers.
I read your article in the thinky tank site. You sound like a pretty good teacher, but not a great teacher. You’re missing something — the most important ingredient in reading instruction. With all of your focus on decontextualized reading passages —selected by testing companies — that may or may not alight upon your students’ interests, on “tasks”, on informational text, on decoding, and on score gaps, you are failing to teach the most important thing.
Reading is fun! Writing is fun!
While you turn reading and writing into a chore for whom you call your children of color, my students are learning that communicating with written language is a powerful joy and a joyful power. I am guiding them through and beyond stories with meaningful, relevant themes. And not only are you drilling and killing, but you’re in a thinky tank advocating for the wasteful spending of more education funds on standardized tests by wealthy testing companies. It’s a multibillion dollar industry funded by school billions of dollars. The students in public schools are being robbed. Your students deserve to have those dollars spent on more and better compensated teachers, not on data collecting for student profiling.
They’re not achievement gaps; they’re funding gaps. They’re not achievement gaps; they’re racial prejudices built on dubious test scores. Every student deserves teachers who not only, like you, care about the students, but also who are in the profession to light the fire of curiosity instead of extinguishing the spark with test prep, what performance tasks are. It’s never too late to change your thinking. Have fun teaching, and earn a decent wage doing it. Drop the tasks. Drop the tests.
One of the things that strikes me about some of these posts is the implied omniscience. One of you knows exactly what I wrote without reading my piece. Another that I have no idea how my students are progressing in reading. A third that there is no joy in my classroom because I focus on “decontextualized reading passages”. Really? Who told you that? The unit I put together on Stephen Curry using Time for Kids and NEWSELA articles brought “joy” and engagement. And the one on dinosaurs, equally so. Drawing and labeling a volcanic eruption or making posters about why koalas need rescuing in Australia are not “drill and kill”. And just today, as my students shared their narratives with the class, there were proud smiles screen-wide. In fact, I don’t need to do test prep with packaged materials because teaching students all year how to analyze and synthesize does the job. And it can be applied to any context. So yes–it may be true that I’m not a great teacher–but until you come into my classroom and watch my kids whistling while they work, you should reserve judgment.
My third grade class ended at three, and I received this email from one of my students an hour later. And that’s what makes it all worthwhile. It’s their validation I want–nobody else’s.
“I miss you and I love to go to your class! I do NOT want to miss a class of yours! unless it is a problem!”
That’s so wonderful!
I am heartened to hear it, and I am glad to read that you do not need standardized tests to be a great teacher, so we can agree it’s well past time to stop throwing money out the school window at the testing industry. Thank you.
LCT,
I agree – there is a disconnect here.
The TEACHING that this person says that she does has nothing to do with testing. Nor did she offer up a single bit of evidence that without her students taking a standardized test, she would have no idea if they are learning what they should be.
In fact, she says just the opposite. She claims to be able to teach them without needing a standardized that that informs her of whether she is a success or failure.
So there is no logic to her argument. If she wants to teach to give joy, bully for her. She still hasn’t offered up a single bit of evidence as to why her students need to be forced to take a standardized test.
Harrett Janetos, why do your students need to take a standardized test for you to be a good teacher?
Do you believe all of your students are the sum total of their standardized test score? Funny how the top colleges now recognize that standardized test scores are not a particularly good way to judge students, yet this teacher is forcing low-income students to believe that their entire value lies in their test score. Why?
All true. Good points, NYCPSP.
“I beg to differ.”
I’m trying to figure out what you are begging to differ with. Please further explain what you disagree with. Thanks!
I read your post and seems to me that what you describe should be going on in the teaching and learning process on a continual basis. I don’t see anything out of the ordinary with what you state, in an easily read article. But, again, I’m not sure what your beef with this post is and how your response here ties in. Please help me understand.
Thanks, Duane, for asking. This is the statement I disagree with:
“Nothing that students do on these exams even remotely resembles what real readers and writers do with real texts in the real world.”
For example, I read text from the left (like this blog) and text from the right (like those from Fordham). I analyze what I read and I synthesize a response to the positions stated, which I sometimes state verbally to colleagues, friends, and family, and sometimes I state in writing, like in this response.
The SBAC is not a timed assessment. If I deem an assessment to be authentic, then I have no problem having it drive my instruction in authentic, engaging ways. Which doesn’t mean, of course, that students aren’t reading and enjoying Dog Man or Diary of a Wimpy Kid, or that they’re not writing and drawing in their journals, or they’re not a part of literature circles or reader’s theater. It means, quite simply, that I agree both as a parent and a teacher that analyzing multiple documents on one topic and responding to them using evidence from the text is an important skill, too important to depend on variability from classroom to classroom to make sure it’s taught. That’s what a standardized assessment can accomplish: consistency.
Unfortunately, we are witnessing the importance of this skill right now as judges from both the right and left examine the documents presented to them about alleged fraud in our election, and they are analyzing these documents and synthesizing an appropriate response (Are you out of your minds!).
I want my students to grow up and be one of those judges, if that’s what they choose. They know that they go public in lots of ways: the soccer coach asks them to play games in front of spectators, they sing at the winter concert, and their drama teacher has them perform in front of an audience. They are well-prepared and confident to take on the task of analyzing articles and synthesizing a response to them–and It’s just no big deal.
Thanks for the explanation!
“It means, quite simply, that I agree both as a parent and a teacher that analyzing multiple documents on one topic and responding to them using evidence from the text is an important skill, too important to depend on variability from classroom to classroom to make sure it’s taught. That’s what a standardized assessment can accomplish: consistency.”
Not sure how you get from the first sentence in that quote to the conclusion you draw about what a standardized test can or cannot do. Again help me out.
Noel Wilson has shown us how using the results of standardized testing, due to the inherent onto-epistemological errors and falsehoods, is “vain and illusory”. Standardized testing is not objective like the proponents insist, nor is it valid. Have you read Wilson’s “Educational Standards and the Problem of Error? See: https://epaa.asu.edu/ojs/article/viewFile/577/700
Any and all involved in education should read that most important educational writing of the last half century. I’d like to hear your thoughts on what Wilson has shown us about the standards and testing malpractice regime.
Harriett Janetos,
“I agree both as a parent and a teacher that analyzing multiple documents on one topic and responding to them using evidence from the text is an important skill, too important to depend on variability from classroom to classroom to make sure it’s taught. That’s what a standardized assessment can accomplish: consistency.”
As a parent, I would have serious questions about any teacher who had no faith in her own ability to teach this but who believes that the student’s score on a single standardized test on a single day will inform that teacher of everything she failed to do because that teacher (presumably Harriett Janetos) had no ability to know whether she was teaching anything during the day to day interaction with that student!
Harriet Janetos, if you have no confidence in your own ability to judge whether you are failing your students as a teacher, why don’t you just download a standardized test, have your students take it, look at the answer sheet, and decide based on your student’s performance if you are an utter failure as a teacher or not?
You don’t need to wait for a yearly “authentic” assessment because you believe your ow abilities to judge a student are so lacking.
If I went to a parent-teacher conference with a teacher like you who told me she had no idea how my kid was doing because only the results of the “authentic” end of year standardized test would tell her, I would want her fired.
Yet you seem PROUD of it!
“They are well-prepared and confident to take on the task of analyzing articles and synthesizing a response to them–and It’s just no big deal.”
You mean it is no big deal as long as they prove it to you based on their performance on a one day standardized test.
If they demonstrate it during the year multiple times, but their one day standardized exam doesn’t show that they did, which do you believe?
If they don’t demonstrate it during the year while you are teaching it, but a standardized test that someone else grades “informs” you that they have this skill, who do you believe?
I don’t understand why you don’t trust your own judgement at all, and while all teachers should be open to being wrong, I have found that teachers that have no self-confidence at all in their own abilities and are so insecure about their own teaching abilities that they believe that anonymous test graders who aren’t paid very well know a lot better than they do are usually not very good teachers.
I have a lot of sympathy for your plight, Harriett Janetos. You obviously know a lot of good things about teaching, but you just don’t seem to trust your own abilities and you need someone else (like one of those people hired from Craig’s List to grade tests) to tell you that you are doing a good job teaching.
Sadly NYCpsp, over the years I have found many teachers with that same attitude, that same lack of confidence in their own assessing abilities.
Duane,
I didn’t see your reply when I posted, so thank you for making that point. And I stand in admiration of your ability to make your points with grace and calm (which I definitely need to work on!)
There is such a logical disconnect between what this person says about teaching, and her absolute reliance on a single standardized test (graded by other people who may or may not be particularly skilled themselves) as the measure of her teaching. I wonder if she even believes it herself, or if she just finds that spewing that illogical rhetoric gets her lots of plaudits and attention from the billionaires who she seems to admire very much for giving her the “tools” to inform her whether she is a good teacher or the lousy one she fears she would be without knowing how her students perform on a single day’s exam graded by someone else. Apparently, without knowing her students’ test scores a few months after they take this exam, Harriett would have no idea how her students were doing, so her gratitude toward those billionaires seems overwhelming.
As a parent, if my kid had a teacher who had no confidence in her own ability to know whether my kid was learning, but she told me that a few months after my kid took a standardized test graded by someone else, she would know, I would want her fired. Seriously.
I read your comment three times, Harriett, took your indignation on other comments along with me, and here’s my take. Reading, to me, is a relatively simple thing that takes time and freedom to choose and make mistakes. It’s truly a wonderful thing. As I read your comment, I read a lot of jargon, “on-level text, assessment, gateway to comprehension, performance task, evidence from these articles to synthesize a response to a given ‘task’, middle and high school ‘ready’, systematic instruction, synthesize a response, achieving proficiency with this ‘performance.'” Basically terms I would never use to describe the joy of reading and learning.
Here’s how I learned to read: I was given books, time and encouragement. The best class I ever had was in 5th grade where all we had to do was pick out any books we wanted to, have a quiet hour to read, and then, the next day, describe to the class what we read. No grades, no assessments, no judgment–unless we didn’t read. I think that’s pretty much close to every American who lived before NCLB, RTTT and DeVosism. What you describe is a career niche for indoctrinators, not teachers. It’s heaven and job security for you, it’s hell for those of us who value education and the joy and promise of reading.
As a librarian my goal was to foster a love of books. While I could keep stats of circulation, the real results come years after the children graduate. Are they still reading? Do they use the public library or download books on their devices. I’m on Facebook with many of my former students and follow some on Goodreads. My success didn’t show up on any standardized test, but positive feedback from these adult readers eight years after my retirement gives me a warm feeling in my heart.
Thanks for sharing that, flos56!
I hope Bob’s brilliant picturing of the flaws in ELA testing can be combined with similar insights into Math testing, then formatted to get a bunch of groups to endorse as send to members of Congress, especially members of the Senate who wrote ESSA and are likely to be happy to keep testing as a major feature in the law.
I have been looking at the money K-12 education lobbies have been spending to keep their agendas in front of members of Congress. I have also been looking at who proposed ESSA amendments that became part of the law, such as perks for nudging the curriculum to STEM subjects and job credentials,. See also this: http://blogs.edweek.org/edweek/campaign-k-12/2020/12/testing-options-biden-education-pandemic.html
Completely right to focus on changing ESSA law. With a re-write or amendment unlikely, the best we can hope for is an explicit relaxation of enforcement. The new USDOE cabinet pick could help here.
If the tests this spring go forward, I hope millions of parents opt out.
Good morning Diane and everyone.
I often wonder how some of the greatest minds in history and people I admire – Emerson, Jung, Hawthorne, Josepeh Campbell, Sarah Alden Ripley, Emily Dickinson, M-L von Franz (just to name the ones I’m thinking of right now) ever became who they were without standardized testing. ???????
Wonderful, Mamie!!! I have often imagined a hilarious scenario in which Rumi or Shakespeare or Blake or Emerson or Whitman or Melville or Edna St. Vincent Millay responded to one of these dumb standardized test writing prompts. Put that in your electronic grader, LOL!!!
Hello Bob,
My husband, a high school English teacher, often says ALL of what you say in your post!!!
My husband also often laments the fact that students are not required to read books in other subjects (Science, Social Studies, etc).
“A Short History of Nearly Everything” by Bill Bryson is my recommendation for every high school student. The history of scientific discovery through the stories (efforts) of the scientists. Easy read, very engaging, and scientifically accurate. Great holiday gift for that special teenager you know.
Rage, that’s a really wonderful book. Here, a couple more science surveys for laypeople that are just wonderful:
Flanagan, Dennis. Flanagan’s Version. Vintage, 1989.
Silver, Brian. The Ascent of Science. Oxford UP, 2000.
As a former consultant item writer for standardized science tests (under NCLB) here is one stone cold, mortal lock conclusion I have arrived at:
Poorly written, age inappropriate, or subjective standards cannot produce valid or reliable test items or test score data. Common Core ELA standards (and their re-branded spawn) were all of these. Worse than worthless; harmful, damaging, and even worse, scarring.
Bad idea messing with the psyche of novice learners with inaccurate feedback. The 21st century testocracy has produced a generation of students who really believe that they. “suck at __.” (Fill in the blank with tested subject of choice)
Regarding CC math, remove the word “subjective” from my conclusion and we’re left with mostly worthless tests/test score data. Talk about a subject that has thoroughly convinced the majority of students that they “suck at math”. Boy have educators failed to self correct on this.
Amen
Standardized testing feeds the test and punish machine.
I have always found it curious and counter productive that standardized reading tests insist on cold reads as a means to gauge student understanding and reading proficiency. Given the importance of prior knowledge and experience acknowledged in current brain development theory, cold passages are a false determinant of reading mastery. Such tests therefore become self-fulfilling prophesies for failure that are used to justify defunding public education.
100%
Always wondered why reading passage topics are not announced in September for spring tests. This would eliminate, to some degree, the advantage of affluence and parental involvement. It would level the playing field so to speak.
Middle class students are much more likely to have prior knowledge that they can access in both fiction and non-fiction selections.
My husband always talked about teaching in the inner city of Buffalo – it’s like trying to put up wallpaper without any glue.
Even if standards and tests were significantly improved, test score data would remain useless for improving (driving) instruction. Incorrect test responses lack useful feedback for improving instruction because we never knew the exact reasons why.
The range of reasons for missed items include:
Poor attendance
Apathy
Chronic inattention/distraction
Self-fulfilling prophesy
Immaturity
Parental influence
Family stressors
Weak subject aptitude
Learning disabilities
Poor effort or attendance during test prep sessions
No-stakes testing
Test apathy/fatigue (science always followed ELA and math)
Brain development delays
Simple confusion and/or misremembering typical of novice learners
Occasional unforced teacher error
Forgot to add the newest reason, courtesy of RTTT and the Duncan NCLB waiver program:
Student revenge (not many but it is real)
Agree. This of course applies to teacher-designed exams as well. That’s why in the normal course of things, they are only one piece of a several-pronged approach to assessing and grading. Tests, homework, attention/ participation in class, and really, all aspects of learning are affected by the things you list. That’s why teachers need to be free to use the full year for teaching responsively to student needs. The annual stdzd accountability machine has eaten a great hole in teaching time, which narrows the curriculum, warps it out of shape, and limits creative approaches to getting it across.
It’s interesting that the testing proponents never actually engage with opponents’ arguments. They simply reassert their claims. This is characteristic not of science but of cultism.
Nor do they dare address the 800 lb. gorilla that’s riding around the testing room on an elephant:
In any given class, all (25ish) students who have received identical instructional and learning opportunities will produce the full gamut of test scores. We have millions of very successful students to look to. So why have the testocrats ignored the traits and habits of successful test takers? Why have they instead tried to use negative test data to “drive” instruction? Rhetorical questions to which every teacher knows the answer. Answers to which those testocrats ignore or denounce because the truth reveals their scam.
This is a wonderful image: the 800 lb. gorilla that’s riding around the testing room on an elephant
Declaring that the educational silver bullet is cast from standards, curriculum, pedagogy, and/or software has been the biggest educational con of 21st century.
and, TESTING (of course)
Exactly. A very lucrative con.
Hello everyone,
I often wonder how some of the most creative, innovative, brilliant intellectuals (and some just brilliant plain old people with everyday lives!) from the past ever did what they did without the help of standardized tests. ???? Thank the universe that people I admire – Jung, Emerson, Thoreau, Sarah Alden Ripley, Hawthorne, Emily Dickinson, Joseph Campell, Rilke, Balzac (just to name the very few I am thinking of now) – never had a standardized test!! How did they learn to read? How did they learn to analyze? How did they learn to write?
The testing mania has been a breathtaking boondoggle. It hasn’t improved outcomes AT ALL. We did far BETTER, I think, when we weren’t wasting all this instructional time on testing and test prep and on pedagogy and curricular materials that have been test-prepified. There was a time before all this rank nonsense, and that time, and its public schools, produced people who built the most successful economy in the world. Go figure.
All true Bob. But I think one of the seldom talked about consequences regarding the test-threaten-and-punish boondoggle has been the degree to which it added an additional layer of tension in the most challenging, high needs schools. This was exacerbated by RTTT and the Duncan (NCLB) waiver program which linked teacher evaluations to test scores. Many schools became exhausting, pressure cooker work environments as threats to achieve the impossible (improve scores) remained relentless.
Absolutely. Horrific!
In real life you would take your time and look up any information you were unsure about, whether it’s the background on a particular subject or the definition of a word. I’m working with my grandson in second grade. His teacher does an excellent job, but I’m there to help, especially with his independent work. If he is having difficulty, I model the appropriate behavior so he can complete the assignment. I also read with him to demonstrate how to read with feeling and expression which he then imitates.
Ultimately, it is the process not an individual answer which is the most important. Trick questions on a test doesn’t provide an accurate measure of a child’s ability. What they do assess is how well a child does on figuring out how to take that particular test.
How wonderful you’re there to enrich your grandson’s learning experience. I know of other grandparents doing this, there must be many. A rare silver lining to the covid instruction situation.
I split my time between my grandson and my elderly mother. I’ve become closer to both which is indeed a blessing. Who knew my retirement would end up this way!
Ok, this is my 3rd time trying to post this comment! The technology gods are tricksters today! Does anyone ever wonder how some of the most brilliant, creative people ever did what they did without having standardized tests? Thank the universe that people I admire – Jung, Emerson, Hawthorne, Joseph Campbell, Rilke, Emily Dickinson, Balzac, Sarah Alden Ripley (just to name a very few of my favorites!) – never had to take a standardized test!! How did they learn to read? How did they learn to write? How did they learn to analyze? I have a few ideas…
I was busy editing a book review that I wrote and I was offline. That’s why your comment was in moderation.
Why WordPress put it in moderation is a question I can’t answer.
Nor can I explain why faithful readers are suddenly unsubscribed.
Sorry for the same post 3 times! I didn’t get a “Waiting in moderation” notice, so I didn’t think these were being posted. Apologies!
WordPress often does this when one includes a list of names, I’ve noticed. Better not start making a list of Ed Deform ghouls!!! LOL.
Kinda related to this: I read the book No Rules Rules by Reed Hastings and Erin Meyer earlier this week for work. As many of you know, Hastings is the founder/owner of Netflix, one of the billionaire deformers, and board member of one of the movement’s greatest scams, KIPP. He has funded and supported, among other things, regimentation, testing, “accountability,” and undermined the professionalism and autonomy of teachers. There are some interesting and potentially valuable parts of the book with respect to businesses, but there were parts where it was hard to keep the contents of my stomach down.
In one section where Meyers cites a study that “the offer of higher bonus led to poorer performance” she concludes, “This finding makes perfect sense. Creative work requires that your mind feel a level of freedom. If part of what you focus on is whether or not your performance will get you that big check, you are not in that open cognitive space where the best ideas and most innovative possibilities reside. You do worse.” To which Hastings added, “People are most creative when they have a big enough salary to remove some of the stress from home. But people are less creative when they don’t know whether or not they’ll get paid extra. Big salaries, not merit bonuses, are good for innovation.”
Imagine that. Why does this not, in Hastings’ world, apply to teachers, public schools, and students’ opportunities? I guess only the Shadow knows.
[One of my biggest of peeves is the use of the jargon of “the (insert subject here) space.” I noticed the use of this beginning in the early 2000s in DC, when people working there would inject it into conversation to make themselves sound smarter, or so they thought. Whenever someone uses “the…space,” it’s like fingernails on the chalkboard to me. They are superfluous word and add nothing. In “the education space” is just education. In “the cancer space” is just cancer. Or as above, “in that cognitive space” is just a doozy. Don’t get me started on the use of the preposition “at” at the end of a sentence!]
The “education market” is definitely a fingernails on chalkboard term. Instead of seeing education as a public service, these profiteers consider education a commodity.
What’s worse is considering the students a commodity.
As Peter Greene (Curmudgucation) says, the testocracy has flipped the script, asking students to work for teachers and schools instead of the other way around. When you look at it this way, it is a more twisted system than the formerly innocuous word “testing” might suggest.
Thank you, Bob Shepherd.
Reblogged this on Crazy Normal – the Classroom Exposé and commented:
The High Stakes Test con game: Nothing that students do on (profitable) high stakes tests even remotely resembles what real readers and writers do with real texts in the real world.
Bob brings the fire. Wow.
I remember being at some “inservice” conference many years ago and a teaching “coach” was trying to explain the standards to the assembled faculty. And I kept raising my hand and saying the whole shebang made no sense to me.
This “coach” finally said I should :”think outside the box.”
Ha, ha,….who was in a box….
Thanks, Bob.
“Never badder than bad cause the brother is madder than mad”
Chuck D, Bring the Noise
LOL
John
During the first or second year of NCLB we had a rep from NYSED describing the test-and-punish policies of this new law. She was describing the requirement for test score improvements (AYP) in each of the different at-risk subgroups. There was a minimum number of students in a subgroup in order for it to be factored into the AYP requirement. In New York it was only 12 students per subgroup if I remember correctly. And schools had show yearly improvement in subgroup scores even though a new subgroup was being compared to the previous year’s subgroup. When she was done, I raised my hand and stated that the sample size for subgroups was too small to draw any significant conclusions, much less suffer the consequences and stigma of being labeled a SUNI school. Her response is etched into my brain: “I don’t give a shit.”
cx. SINI school (School In Need of Improvement)
A long, long time ago, Rage, I went to a social studies conference out in Rochester, NY. I remember this wizened old teacher with a long gray beard laying into a NYS Ed Dept rep about some ridiculous question they’d put on a recent Global Regents Exam. He was absolutely correct. And the woman from State Ed. came back with a smarmy answer, like, yeah, it was a problem question but she sure bet schools all over the state were going to teach that trivial fact the next year.
(I will say that was back when the bureaucrats actually could and would venture out of their massive concrete bunkers up in Albany.)
Now that I’m the teacher who is old and gray I hope I can be at least half the pain in the ass that guy was.
Yeah, AYP…the cult of progress…people being treated like machines.
Despite it all, have a great weekend.
Here’s a quick MC item for any teacher who taught in a “labeled” NY school during NCLB:
1) Under the NCLB act, NY schools that failed to meet AYP goals
were branded with the “SINI” label (School In Need of
Improvement). The “SINI” label signifying underperformance was
eventually changed to the “F__CUS” label.
1) A
2) E
3) I
4) O
5) U
Great post, one of my favorites from Bob Shepherd’s Praxis.
Bob’s why? questions are rhetorical. As with so many “why’s” about repeated nonsensical, socially-harmful actions, the answer is “follow the money.” Obama/ Duncan’s doubling-down on high-stakes ed-accountability for public schools was accompanied by generous start-up cash aimed at expanding privately-managed, publicly-funded charter schools. Labeling traditional publics “failing” expands the market for alternatives whose only distinction in “achievement” [as measured by inauthentic tests with results doctored via cut-scores] is funneling public funds into corporate hands. Meanwhile the expensive accountability systems reqd for all pubschs (incl most charters) lines another set of corp pockets. Privatization of the public commons all the way around.
Under Trump/ DeVos, all pretense that this is about accountability is abandoned in the push for voucher schools. Maybe 5 out of 23 voucher states require state assessments. 3 require no testing. The rest want a nationally-recognized norm-referenced exam— so, no apples-to-apples w/state exams. Articles on what kind of ed-accountability they should have are myopically focused on how to make ‘ed-scholarship’ programs viable by roping in sufficient privschs: higher-qual privates don’t need to waste $ on these tests, don’t want to sacrifice innovation and creativity on teaching to the test– let them use their norm-referenced tests! The irony that these arguments apply equally to the state’s 88% pubsch students seems to escape them.
Many of these comments dance around Bob’s main point: that the ELA tests are based on a misunderstanding of what good reading is. I am a highly-educated middle aged avid reader, and yet I am not ashamed to admit I would have great difficulty describing how an author’s use of figurative language affects tone, and half of the other things the Common Core standards demand. I’m not ashamed to admit it because these are NOT IMPORTANT SKILLS. They’re pretentious and arbitrary benchmarks established by people who don’t really understand what reading is about. We need to make kids good at comprehending texts, not performing brain-paining analytical gymnastics. How do we do that? By imbuing their minds with a solid foundation of background knowledge, the key ingredient in reading comprehension. We teachers are so far off base. I worry about America’s future. Meanwhile China and other countries have roundly rejected our stupid education ideas.
Ponderosa,
Yes! 100% but the tests cannot be fixed without fixing the standards. Instead states like CA and NY (plus 18 others) doubled down on this inane attempt to teach soft critical thinking and problem solving skills, minus the requisite knowledge to do the thinking by adopting the Next Generation Science Standards. The pass rate on the new CA NGSS tests are disastrous and completely misconstrued by those who don’t know better. Just don’t blame the tests.
We are so far down the “anti-knowledge” rabbit hole with this ass backwards approach that recovery may be impossible.
Ponderosa,
I agree with you there for sure. It is strange that people who write tests for ELA think that students should have understanding of arcane literary devices that are of no importance. What matters is knowing and understanding the words on the page, not inferring the author’s tone and intent. I recall posting here a few years ago about a poet who saw her poems on a standardized test, with questions about her intent. She said the testmakers completely misunderstood her poetry.
Misunderstood poem aside, test writers are forced to write crappy items only because they are required to follow crappy standards.
The really insurmountable problem for ELA test developers using Common Core standards is that they have to write objective, MC items for highly subjective standards. The two are fundamentally incompatible. As a consultant test writer, the science standards I wrote for (a dozen different states) were usually objective and direct and the resulting items would be fair and valid. Occasionally I’d get a standard that was poorly crafted or very vague and the result was a very shaky test item that I hoped would not make the cut. Believe me, test writers are completely constrained by the standards and how well they match the required formats: MC and ER. This is why we hear far fewer complaints about the CC math tests; the math standards are mostly objective and result in more reasonable test items. CC math standards did get into trouble whenever they were semi-subjective, confusing, or were written above grade level. Here’s a grade 4 CC math standard; just for kicks, try writing a high quality MC item that adheres to the intended learning.
Generate a number or shape pattern that follows a given rule. Identify apparent features of the pattern that were not explicit in the rule itself. For example, given the rule “Add 3” and the starting number 1, generate terms in the resulting sequence and observe that the terms appear to alternate between odd and even numbers. Explain informally why the numbers will continue to alternate in this way.
Sometimes, knowing the name of a thing and the attributes of things that have that name can help with seeing what’s going on and being able to reproduce it. So, for example, you can teach kids what chiasmus is and give them some examples
Never let a fool kiss you or a kiss fool you.
The first shall be last, and the last shall be first.
and so on.
My point was not that we should eliminate such teaching but that that this stuff is incidental to greater goals–arranging for kids to have transformative vicarious experiences in works of imaginative literature, having them engage with the concepts in works of informative and persuasive literature. What’s the value of this? is always the kid’s question, and it’s a perfectly valid one. If a student is asked to read FDR’s Four Freedoms speech, and all that’s done with it is to look for examples of parallelism in it, well, that’s a travesty. What really matters is what the man had to say. And it matters a lot. That’s all I meant by that line in my essay.
LOVE the “Never let a fool kiss you or a kiss fool you!”
Does the latter include Hershey’s Kisses?
(Just meant as holiday humor.)
I have been sharply critical of the St. Louis Post Disptach for more than a decade…Kevin McDermott is on the editorial staff….I complimented him for an excellent article he offered about Senator Hawley….”Hawley’s populist demagoguery is a smoother form of Trumpism. That’s dangerous.” I told him there is a connection with Bob Shepherd’s article, and gave him the link…St. Louis has fallen hook line and sinker for what Shepherd explains.The pd has a new black writer on the editorial staff….antonio french. I suggested that he talk to antonio, and the person who had to be eliminated because of his opposition to charter schools…..Peter Downs, who has also written about testing. It is an awful lot to ask….but if anyone could urge Kevin McDermott to take a look at what Shepherd wrote—he might be open to taking a look at it.
Kevin McDermott is a member of the Post-Dispatch Editorial Board.
As a parent who was also an educator, the main thing I wanted to know was whether my child was on level, below level, or above level in any particular subject. The grade was really insignificant in the early grades and more a reflection of my children’s work ethic in the upper grades. My two oldest daughters realized early on that they could work hard and not get that A, but could coast along with a lot less work and get a B. They were good with the B. I just wanted them to do their best. My younger daughter had a processing problem and had to work twice as hard as her sisters to get that same B. While they only had average scores on the SAT, they all graduated from college and are successful in their chosen careers.
As far as my grandson, I’m not his teacher but I know he’s on grade level in reading and advanced in mathematics. It amazes me that he can figure things out, sometimes without any instruction at all. He’s finished the assignment (correctly) before the teacher is even done with the first problem and I have to find blank optional pages in the workbook to keep him focused. She’s talking about the number 643 and he’s deciphering the number 1,643,576. It’s not a matter of memorizing math facts – he gets the concepts, sometimes figuring them out intuitively. Of course, it’s only second grade, but I predict he’ll continue to be a whiz in math. I just hope I can keep up,
This was true with the SAT and all the other standardized tests through the years. Testing of this sort is cheap and clear. What it tells is how well students take tests. How people use it is to justify neglecting the education of the poor and not white. It was and is appalling. This writer is not the first to say it and will not be the last. Monetizing this trickery is big business. When I taught in Pine Bluff, Arkansas, in a district taken over by the state for failure, teaching standardized tests was the only education the school and the district cared about, encouraged, or tracked. The students were as students anywhere except they were poor and African American.