As readers are aware, Congress is considering reauthorization of No Child Left Behind, which should have been reauthorized in 2007. One of the most contentious issues is whether to retain or modify the federal mandate for annual testing. Some have proposed grade-span testing as an alternative, since annual testing has caused some schools to spend a disproportionate amount of time on test preparation. Some would like to see the federal trying mandate eliminated altogether, with federal money used for equity rather than standardized testing (I’m in the third camp but would find grade span testing an improvement over annual testing).
Recently a dozen civil rights groups released a statement criticizing parents who opt out of annual testing. The Network for Public Education responded in disagreement in a statement written by teacher Jesse Hagopian and the NPE board. Mark Tucker wrote a post disagreeing with the civil rights groups, saying there was no evidence that annual testing helps poor and minority children and some evidence that it harms them by narrowing the curriculum to test prep.
Kati Haycock, leader of pro-testing Education Trust (which helped to draft NCLB), responded angrily to Tucker.
Here, Mercedes Schneider challenges Haycock for her defense of annual testing. Schneider says that Haycock failed to refute Tucker’s evidence and instead went on a rant.
Schneider writes;
“In her June 4, 2015, Education Post rebuttal, Haycock jumps out of her daytime-TV chair, knocking it back as she rushes forward to get in Tucker’s face while declaring that she, “even a white girl,” can register what is Tucker’s obvious insult: That the civil rights community could possibly be injuring children by insisting upon annual standardized testing.
“No such drama was necessary. All Haycock had to do was refute Tucker’s evidence.
“She did not.
“Instead, she goes on to write (in the $12 million, Walton-Broad-Bloomberg-funded, corporate-reform Education Post) that she– the white girl– is there to call Tucker out on behalf of a group of 12 civil rights organizations that she admittedly did not join with in their May 5, 2015, formal declaration against opting out.”
Watch and read the verbal fisticuffs. It might be funny if it were not so sad. The evidence matters.
PS: Marc Tucker responds:
“Tucker told Morning Education that Haycock is “just plain wrong.” The civil rights community is not as united on testing as many think it is, he said, citing a recent op-ed [http://bit.ly/1BKzpI3]. “I actually laughed when I saw it, to tell you the truth,” Tucker said. “What’s important to me here is not overriding the civil rights community, but persuading people in it that they have misread the situation.”
.
Diane Ravitch's blog 
I wish I had time to take on Tucker’s “evidence” — because it mostly misses the point of annual testing, which is to give parents and taxpayers information about how their students are doing. Tying the tests to teacher evaluations is also a strawman since, in most cases it represents but a small part of the evaluation process. In my small school district, where I served on the BOE for five years during much of the NCLB rollout, I noted a remarkable turnaround; not in test scores but in the engagement teachers were forced to make with minority students. For the first time, thanks to the racial and demographic breakdown in test results, parents and taxpayers saw the amazing discrepancies in performance outcomes. And that spotlight made administrators and teachers much more diligent in their teaching and in their search for improvement models. While the district, like most public schools, remains hobbled by bad pedagogy and non-existent curricula, it is a much changed school culture. Haycock is absolutely right: killing the annual testing will only help to turn the lights back off for minorities. –peter m
LikeLike
Let’s suppose, for the sake of argument, that testing does, in fact, give parents and taxpayers information that they had no way of knowing before. So what has been done with that information? When has that information ever been used to help poor students, students of color, students learning English or students with disabilities, rather than just close/turn around/charterize their schools? Give me some specific examples of situations in which test data has led directly to an improvement for the students involved. Heck, I’ll even let you use improvements in test scores. However, the condition is that it has to be the same students. “PDQ school turned around or charterized and has a completely different student body and now the test scores are fabulous!” doesn’t count.
LikeLike
I thought I answered the question about what was done with the information: administrators, teachers, parents, and citizens all started paying attention to what was taught in the school, how it was taught, and how to improve. If you don’t make it into the hyper-partisan political thing it has become, it’s a big win-win for districts. I’m not a big fan of standardized tests (it would be much better to put the horse in front of the cart and test on a written and taught curriculum), but for now, in the absence of good curricula, it’s all we got. All the ruckus over the Common Core, IMHO, is a big distraction, but it has at least, made more people aware of what is actually being taught (and not taught) in our schools. Information is always a good; properly gathered and used information is even better.
LikeLike
“…but for now, in the absence of good curricula, it’s all we got.”
Of course, testing is a large part of the reason the curricula sucks….
LikeLike
I don’t know that testing per se causes the problems with curricula. In the absence of curricula, tests rush in to fill the void; but they can’t fill it. It used to be that our educators shared a very similar value system, including values of what should and shouldn’t be taught in classrooms, what books to read, what knowledge to know, etc. (i.e. the curricula). We were able to have such a national curriculum by dint of school cultural homogeneity. That school culture started breaking apart with Dewey, but political skepticism about top-down government mandates is still quite powerful; an absolute paranoia about “national curriculum.” (The feds are forbidden by law from writing curriculum, which is why we get such strange and contorted programs like the Common Core.) So, we no longer have a national curriculum, but, thanks largely to Dewey and the child-centered classroom movement, we no longer have curricula even at the local level. They might call it a curriculum, but it’s nowhere near the full-throated core-subject, content-based curricula from 50 and 60 years ago.
LikeLike
When teachers’ evaluations and schools’ fates are determined from test scores, test scores are going to drive curriculum. What gets tested gets taught because no one wants to be the one (or one of many) left without a chair when the music stops. Of course, this mainly applies in poor and minority areas because the tests favor affluent whites whose children will, by and large, do well even without being taught to the test.
LikeLike
Dienne, I think you’ve got the cart and horse reversed: the only reason that middle- and upper-middle (and rich) kids do better than poor kids is that the former have had access to lots of background knowledge, which comes in the form of lots of reading and exposure to content-rich vocabulary. If our schools offered such a content-rich curriculum in the schools, even the poor kids would pass those tests. But the point is that a good curriculum trumps the standardized tests. If teachers spent their time lobbying for (or creating) content-rich curriculum, they wouldn’t have to teach to the test, which is, as you all know, a recipe for disaster.
LikeLike
“But the point is that a good curriculum trumps the standardized tests. If teachers spent their time lobbying for (or creating) content-rich curriculum, they wouldn’t have to teach to the test, which is, as you all know, a recipe for disaster.”
pbmeyer –
In case you aren’t aware, teachers have lost their agency. They no longer get the opportunity to create or to teach content rich anything. They are handed a script and everyone is expected to be literally on the same page each day. The piles of required “documentation” further rob teachers of the time needed to pay attention to the needs – social, psychological, physical or academic of the kids they serve. “Good curricula” i.e. that developed by the teacher, for the students, is forbidden by CCSS, PARCC, and SBAC. Of course, teachers can blithely ignore these mandates and do what they know is best for their students – provided they are independently wealthy or are just doing their jobs for pin money.
LikeLike
Sorry, Christine, but that’s just not true that teachers have no “agency.” See Rick Hess’ “Cage Busting Teachers.” I’ve visited plenty of schools where teachers have lots of agency. Obviously, it helps if they have cooperating administrators. What they don’t have is “autonomy,” the loss of which has been caused, in part, by the failure to agree on what should be taught in our schools. A curriculum is a common instructional program — it’s not a blueprint to let teachers teach what they want. If you don’t believe that a common instructional program — this is consistency and coherence across grade levels and subjects — is of value, then, in my opinion, your school and/or district will have trouble, because kids need consistency and coherence. But if you agree that a common instructional program is of value to kids, then it’s a matter of making making the “script” work; it’s not easy, but with a collaborative effort, it’s possible.
LikeLike
Test prep as curricula, usually dictated from above, is a relatively new thing and is in direct response to the increasingly high-stakes nature of standardized testing, especially for the schools.
Also, it’s not just a matter of what happens in schools. Studies have shown that poor kids, on average, show up at school having heard millions fewer words than their affluent peers. Obviously poor kids don’t get the kinds of travel and museum experiences as affluent kids. They’re less likely to have books in their homes and people to read to them. So poor kids show up from day one at schools at a significant disadvantage, academically speaking, to their affluent peers. The tests are designed for affluent kids who have had the being read to, going to museums, etc. kinds of experiences, so if poor kids – and the schools that serve them – are to have a chance, intense test prep is necessary or else their teachers will be blamed for their “failure” and their schools will be closed.
You live in a fantasy world if you think test data is going to somehow wake up the elites to the problems of poor and minority students such that they’ll come swooping in with resources and support. They’ll come swooping in all right, as we’ve already seen, with charters and vouchers and other ways to transfer public assets into private hands. Never let a crisis go to waste. You refuse to see that at your own peril.
LikeLike
Who said anything about convincing the “elites”? The testing data woke up parents, teachers, administrators, and average citizens. And then when they realized that there was no law against have a good curriculum, they started working on that. A big part of this is that these stakeholders began to realize that they had a great deal of agency–you can improve lots things in your school and still follow all of the mandates (including the stupid ones).
LikeLike
“Who said anything about convincing the “elites”?”
Who do you think is supporting, funding and politically driving these tests? And why do you think they’re doing it? A touching concern for poor and minority kids? Wake up.
LikeLike
That’s the kind of comment that ends most school improvement efforts. Too bad.
LikeLike
Dienne: I guess I am just seeing this from a different POV but just by what I have read in this thread…
I took the leading comments as a candid, if very painful and self-wounding admission, that the adults with political, financial and social heft in the community—first and foremost, school leaders with BOE heads bearing the greatest burden of responsibility—ignored countless other signs long long before the test scores came out—
That they had forced colossal failure on students, public school staff, and the entire community.
You don’t need test scores to tell you that you have committed educational malfeasance. It’s not a secret that stays hidden until, by the magic of the numbers, the curtain is drawn back and the truth stands revealed for all to see.
I applaud the author of the comments for admitting to being a major part of the problem but I hope that this will lead to a reconsideration of corporate education reform and its measure-and-punish use (and abuse) of standardized testing.
Just my dos centavitos worth…
😎
LikeLike
I would refute the notion that the teachers of minority students having malaise is the problem. The problem is the distribution of resources to minority students from the district or the state. Will testing bring the best resources, facilities, and teachers to minority students? By what mechanism?
“I noted a remarkable turnaround; not in test scores”
Tests as a motivational tool, not valid as a measurement device?
Interesting.
LikeLike
I certainly didn’t — and wouldn’t — use the word “malaise” to describe teachers. Suffering from PTSD because of the difficulty of their jobs, perhaps, but not malaise. The bureaucracy has a malaise problem. The “thought world,” in E.D. Hirsch’s opinion, has a malaise problem. Indeed, the bigger problem is institutional: the wrong ideas about what a good school is and the poor implementation of the good ideas you might have. As mentioned above, I’m not a fan of standardized tests, and certainly not annual ones (with the results coming out 3 or more months after kids move on), except as a measure of how a school or district is doing in the broadest sense. And if you break out the results by race and demography, the results can be pretty striking. And yes, the results should really motivate the school stakeholder community to do something!
LikeLike
As an ESL teacher, I have also heard the “at least are language minority students won’t be forgotten” in defense of annual NCLB testing. To tell the truth, I never saw them forgotten in my district before NCLB. The only change I noticed was that the staff started to be less willing to be innovative due to concerns about standardized scores. The curricula started to narrow, and teachers started to get called into the principal to discuss scores. Now from what I understand from former colleagues (I’m retired), total paranoia has paralyzed any attempt to do anything new or exciting. Innovation is dead due to the test and punish agenda, and everyone is in survival mode. “Thank you, Mr. Obama!”
LikeLike
retired teacher raises a wonderful point: how NCLB so narrowed the focus of the standards (ela and math) that the disabled kids got ignored. In fact, lots of kids got lost by this crazy focus on ela and math. But that’s not the annual testing requirement’s problem per se. As has been argued by others, schools with a rich, content-based, core-subject curriculum did all right with the standardized tests. Those without such a curriculum — and without the leadership to implement it — were left to teach to the test, which is a terrible waste of money and teacher talent. A good liberal arts curriculum would lift all boats, including those of the gifted and the disabled.
LikeLike
Excuse me, you said ESL…. Sorry, but I my comments below would apply to ESL kids as well.
LikeLike
My school had a comprehensive curricula, and our students did well on standardized tests because of it; in fact, we were a Blue Ribbon School in 2005. However, our ELLs as well as many classified students still lagged behind. Many of the ELLs, if they have enough time (5 to 7 years), can do much better, but some of the classified students had disabilities that didn’t dissipate with time. The pressure this students feel to achieve crushes their souls, and it is counter productive to blame either students or teachers for the scores.
LikeLike
Peter, I have to disagree with you that “the point of annual testing… is to give parents and taxpayers information about how their students are doing.” As a taxpayer I get test scores for my school and my school district – and those scores are not disaggregated enough for me to get, as a taxpayer, much useful information from them. A broad overview, yes, which may or may not give me what I need to know about how “good” the school is or how “effective” the teachers are….so not really useful to me as a taxpayer.
As a parent, I get ongoing and up-to-date and specific information from my kids’ teachers, so I’m able to stay on top of their progress; even if PARCC actually *does* successfully manage to someday get their results out by the end of the school year (as opposed to 5-7 months later LOL), that only tells me about things it’s too late to fix that year, so….useless in that regard either.
The only point with which I’ll agree is that the testing did shine a light on the achievement gap, on cases (and there were admittedly too many) where poor students and students of color were indeed suffering from low expectations by teachers and staff. But after a decade and a half of testing every child every year, has that gap narrowed in any meaningful way? There was already some narrowing happening BEFORE NCLB, and now at least where I live now, the gap has either stalled out or widened, depending on the individual school/neighborhood. Where is the closed gap? Where is the increased achievement by minority students? Where is the shining light of knowledge in those communities, where are the promised legions of poor children of color going to college and escaping poverty?
*crickets*
LikeLike
crunchydeb, all I’m saying is that these annual tests are a necessary, but certainly not sufficient, part of the school program. There are plenty of reasons why schools aren’t improving, but you can’t blame it all on testing. As I said earlier, in my district, the tests were incentives to improve — and because we embarked on that improvement in a relatively (not perfect) collaborative way, we have seen test score and graduation rate improvements. It’s a longterm process.
LikeLike
Pbmeyer,
There is no good argument for annual standardized testing. The tests provide no useful information. They have no diagnostic value. Teachers learn nothing about their students. No high performing nation in the world has annual high stakes tests. If this is such a grand idea, why don’t private schools do it? After nearly 15 years of it, parents are sick of it. I would not be surprised to see opt outs double next year.
LikeLike
I would like to challenge pbmeyer2014 to identify the name of the district where “testing data woke up parents, teachers, administrators, and average citizens.”
That claim may be somewhat difficult to verify without conducting a survey of the parties involved way back then, but it should be relatively easy (I would hope) to determine if it is true that “the tests were incentives to improve — and because we embarked on that improvement in a relatively (not perfect) collaborative way, we have seen test score and graduation rate improvements.”
LikeLike
BC, it is reformster hype
LikeLike
Thanks, Diane, for so bluntly just stating the obvious. Now that I’m pretty well versed in reformster-speak I detected all of the tell-tale signs. I just decided to go the more oblique route, which would predictably be followed by…crickets…thereby proving that point. Sincere thanks for pointing that out Diane!
LikeLike
Crunchydeb, the achievement gap was well known before NCLB. What is the good of shining a light on the gap, testing it year after year, and doing nothing to close it? Like reducing class size?
LikeLike
So the achievement gap doesn’t exist in your school anymore ? It was all about teachers not paying attention to minority children before ? It took testing and grading teachers based thereon to change their racist practices ? [So, what I am saying here is you have a credibility gap — your story is inherently incredible. I teach at an integrated school — my children attend integrated schools….your story is ridiculous.]
LikeLike
Sorry, MomintheMidwest, but I was there. It’s not ridiculous.
LikeLike
Was this a real public school, or some kind of voucher or charter school? In my experience in a diverse school, teachers are far LESS likely to collaborate or support those with disabilities or ELL students, because those students drag down test scores, which now affect our evaluations. In Utah, if we are found to be “ineffective,” we cannot get a step increase or any other kind of salary increase. I teach a lot of students with special needs in the general curriculum, and I know that they will struggle with these tests, and that my evaluation will probably drop as a result.
LikeLike
It would be nice if someone, somewhere produced one example of a child of any race who was helped because of high-stakes standardized testing, or barring that, one example of a school which wasn’t already known from preexisting measures as lagging behind before high-stakes standardized testing began.
Really, this is all a huge farce and even the proponents of high-stakes standardized testing know it.
LikeLike
Thank you, Steve Magruder, the misuse of high stakes standardized testing distorts the purpose of education and miseducates. It is a farce and a tragedy.
LikeLike
“It is a farce and a tragedy.”
And it is EDUCATIONAL MALPRACTICE and UNETHICAL to boot!
LikeLike
@pbmeyer, is your view that education must be standardized around a well-defined common culture and we must set aside the errors of Dewey’s progressive education in order to improve education? If so, that may reflect one view of the aims of education.
LikeLike
Not true here. Test scores are 50% of a teacher’s evaluation. Clearly your anti-teacher, anti-public school bias clouds your judgment (“bad pedagogy”?). Using tests as a means to punish teachers into performing some action is indeed a strange management technique but I often see from mindsets rooted in misanthropy. The idea is people cannot be lead, only coerced into achieving a goal. But I also know nothing will change your mind, even attempts at explaining reality or sharing experience.
LikeLike
The Education Trust is a wholly owned subsidiary of billionaires who want market-based education and wrap themselves in the the PR of civil rights in addition to using professional PR people like Katie Haycock to heighten the hype about the necessity of test scores to secure fair treatment for students when, in fact, test scores assure students who do poorly will get a double or triple dose of that mind-poisoning, and spirit-killing fare under the banner of education.
LikeLike
I didn’t opt out, but I will next year if the Obama Administration or the ed reform “movement” (but I repeat myself!) betray their promise not to use these test scores to attack public schools or promote their preferred sector(s).
I’ll do it just because that’s a betrayal of the trust of students, teachers and public schools and there should be consequences for assurances made in bad faith.
The only leverage I have is denying them one set of test scores because listening to them the only time public schools are mentioned at all is when they’re haranguing or scolding us about turning our kids over for testing. It’s really that simple.
LikeLike
Laura H. Chapman: quite so.
And those who claim that standardized testing is essential for bringing to light the evil failures of public schools:
[start]
1. How many schools will NCLB-required testing reveal to be troubled that were not previously identified as such? For the last year or so, I have challenged defenders of the law to name a single school anywhere in the country whose inadequacy was a secret until yet another wave of standardized test results was released. So far I have had no takers.
[end]
[Alfie Kohn’s contribution, p. 86, in MANY CHILDREN LEFT BEHIND: HOW THE NO CHILD LEFT BEHIND ACT IS DAMAGING OUR CHILDREN AND OUR SCHOOLS, 2004]
The above was written in 2004. Anyone who needed and needs to know how well students and public school staff were faring (in the full sense of phrase) by their test scores would have been and are ignorant of failings only through ignorance and willful neglect.
Test scores tell you something about students hitting numerical goals on a single extremely narrow task called test-taking. By their nature those tests are inherently imprecise, measure very little if anything, and are increasingly used and abused to hurt those subjected to the measure-and-punish regimen of the self-styled “education reform” crowd.
And those test scores count very heavily against teachers, students and public schools. Why? Precisely because they are consciously used to mathematically intimidate people and to mislead through the use of descriptive/summary statistics. From the same book quoted from above, George Wood, at the time a principal of a HS (p. 42):
[start]
School people are no fools. Tell them what they will be measured on and they will try to measure up. What this has meant for the curriculum and the school day is that test preparation crowds out much else that parents have taken for granted in their schools.
[end]
But surely the “multiple measures” crowd aka (rheetorically speaking) “the new civil rights movement of our time” couldn’t be so narrow-mined and lazy as to reduce the incredibly rich experience of genuine learning and teaching to a standardized test score?
Just a few examples from one well-conducted study.
[start]
These and other HISD teachers also noted that their supervisors were skewing their observational scores to match their value-added scores given external pressures to do so…
One teacher stated:
Here’s the problem. No principal wants to be called in by the superintendent or another superior and [asked]: “How come your teachers show negative growth but you gave high evaluations on them? Are you doing your job? I don’t understand. Your teacher shows no growth but you have [marked them] as exceeding expectations al up and down the chart?” Now it’s not just this [sic] data of there that’s gonna harm us, it’s the principals [who are] adjusting our data of there to match EVAAS®. So it looks like they’re being consistent. …
Another teacher agreed: “Well my evaluation were fine, but of course now they have to make the evaluation match the EVAAS®. We now have to go through that”…
Another teacher wrote: They’re not about to go to bat [for us, although] a few of them will. But most of them are going to go in there, and they’re going to create a teacher evaluation that reflects the [EVAAS®] data because they don’t to have to explain, again and again, why they’re giving high classroom observation assessments when the data shows [sic] that the teacher is low performing.
[end]
[Audrey Amrein-Beardsley, RETHININGK VALUE-ADDED MODELS IN EDUCATION, 2014, pp. 44-45] [EVASS®: Education Value-Added Assessment System; HISD: Houston Independent School District]
And to conclude: want to see what a long and rich history of rigorous standardized testing does? Yong Zhao, WHO’S AFRAID OF THE BIG BAD DRAGON: WHY CHINA HAS THE BEST (AND THE WORST) EDUCATION SYSTEM IN THE WORLD (2014).
For those that are for social rigor mortis, er, rigidity, and merit/demerit by birth and parents’ social position, then go all in for standardized testing.
But be honest. So say. And don’t be afraid to tout your Marxist fundamentalism:
“The secret of life is honesty and fair dealing. If you can fake that, you’ve got it made.”
Because when it comes to Groucho, the rheephormsters don’t miss a beat…
😎
LikeLike
Is the villain the average teacher, or the funding mechanism?
How much funding would it take to get 90% of poor and minority up to high standard?
If there were beautiful facilities, small classes of 12 or 15 students, highly paid and highly trained and experienced teachers, would that work? How much would that cost? That might be what it takes.
LikeLike
This area of the country was really hit hard by the Wall Street crash. We had 16% unemployment. I have never seen it so high. It was truly frightening. They were feeding kids at the library- people who had never been in such bad shape- they had just been keeping their heads above water and they just needed one push to drown.
Couldn’t one see if there was any effect on scores there? You’d have a “before” and a “during” and an “after”, all other things being equal. How big an effect does family financial security have on kids? What has to happen to fill the hole so kids stay even? It had to be enormously stressful for them. I’d be curious if it showed up.
LikeLike
Because it has never been tried, we don’t know. Except, that in some cities (mine being one) one could make that comparision between existing schools. Except that it’s a political non-starter.
LikeLike
It would probably go a long way with parents if “the movement” would admit they went nuts with testing and compromised on ranking teachers based on test scores since that’s what drove the increased testing. They’re not making the crucial connection between their agenda for teachers and their agenda for students. Students are taking on the burden of the teacher ranking schemes. They’re the people who are over-tested in pursuit of this.
But in order for that to happen a lot of fancy people would have to admit error and make a concession so we all know THAT’S not happening 🙂
LikeLike
Unfortunately, in the country we live in today, sensible proposals like yours are considered dead last, if at all.
LikeLike
Holy smokes. “In her June 4, 2015, Education Post rebuttal, Haycock jumps out of her daytime-TV chair, knocking it back as she rushes forward to get in Tucker’s face while declaring that she, “even a white girl,” can register what is Tucker’s obvious insult: That the civil rights community could possibly be injuring children by insisting upon annual standardized testing.
I guess that gives Kati ‘street cred”. It doesn’t. What I see is another white person screwing stuff up. I know, cause I’m white and I taught 23 years in the city of Washington, DC.
LikeLike
At the Truthout website today, Molly Jane Kneful presents strong counter-arguments to DFER’s advocacy of standardized testing.
LikeLike
THE strongest counter arguments to standardized testing (and its flip side of the coin educational standards) is made by Noel Wilson in his never refuted nor rebutted treatise showing all the errors and falsehoods which result in the COMPLETE INVALIDITY of the the whole process. Read and comprehend Wilson’s “Educational Standards and the Problem of Error” found at: http://epaa.asu.edu/ojs/article/view/577/700
Brief outline of Wilson’s “Educational Standards and the Problem of Error” and some comments of mine.
1. A description of a quality can only be partially quantified. Quantity is almost always a very small aspect of quality. It is illogical to judge/assess a whole category only by a part of the whole. The assessment is, by definition, lacking in the sense that “assessments are always of multidimensional qualities. To quantify them as unidimensional quantities (numbers or grades) is to perpetuate a fundamental logical error” (per Wilson). The teaching and learning process falls in the logical realm of aesthetics/qualities of human interactions. In attempting to quantify educational standards and standardized testing the descriptive information about said interactions is inadequate, insufficient and inferior to the point of invalidity and unacceptability.
2. A major epistemological mistake is that we attach, with great importance, the “score” of the student, not only onto the student but also, by extension, the teacher, school and district. Any description of a testing event is only a description of an interaction, that of the student and the testing device at a given time and place. The only correct logical thing that we can attempt to do is to describe that interaction (how accurately or not is a whole other story). That description cannot, by logical thought, be “assigned/attached” to the student as it cannot be a description of the student but the interaction. And this error is probably one of the most egregious “errors” that occur with standardized testing (and even the “grading” of students by a teacher).
3. Wilson identifies four “frames of reference” each with distinct assumptions (epistemological basis) about the assessment process from which the “assessor” views the interactions of the teaching and learning process: the Judge (think college professor who “knows” the students capabilities and grades them accordingly), the General Frame-think standardized testing that claims to have a “scientific” basis, the Specific Frame-think of learning by objective like computer based learning, getting a correct answer before moving on to the next screen, and the Responsive Frame-think of an apprenticeship in a trade or a medical residency program where the learner interacts with the “teacher” with constant feedback. Each category has its own sources of error and more error in the process is caused when the assessor confuses and conflates the categories.
4. Wilson elucidates the notion of “error”: “Error is predicated on a notion of perfection; to allocate error is to imply what is without error; to know error it is necessary to determine what is true. And what is true is determined by what we define as true, theoretically by the assumptions of our epistemology, practically by the events and non-events, the discourses and silences, the world of surfaces and their interactions and interpretations; in short, the practices that permeate the field. . . Error is the uncertainty dimension of the statement; error is the band within which chaos reigns, in which anything can happen. Error comprises all of those eventful circumstances which make the assessment statement less than perfectly precise, the measure less than perfectly accurate, the rank order less than perfectly stable, the standard and its measurement less than absolute, and the communication of its truth less than impeccable.”
In other word all the logical errors involved in the process render any conclusions invalid.
5. The test makers/psychometricians, through all sorts of mathematical machinations attempt to “prove” that these tests (based on standards) are valid-errorless or supposedly at least with minimal error [they aren’t]. Wilson turns the concept of validity on its head and focuses on just how invalid the machinations and the test and results are. He is an advocate for the test taker not the test maker. In doing so he identifies thirteen sources of “error”, any one of which renders the test making/giving/disseminating of results invalid. And a basic logical premise is that once something is shown to be invalid it is just that, invalid, and no amount of “fudging” by the psychometricians/test makers can alleviate that invalidity.
6. Having shown the invalidity, and therefore the unreliability, of the whole process Wilson concludes, rightly so, that any result/information gleaned from the process is “vain and illusory”. In other words start with an invalidity, end with an invalidity (except by sheer chance every once in a while, like a blind and anosmic squirrel who finds the occasional acorn, a result may be “true”) or to put in more mundane terms crap in-crap out.
7. And so what does this all mean? I’ll let Wilson have the second to last word: “So what does a test measure in our world? It measures what the person with the power to pay for the test says it measures. And the person who sets the test will name the test what the person who pays for the test wants the test to be named.”
In other words it attempts to measure “’something’ and we can specify some of the ‘errors’ in that ‘something’ but still don’t know [precisely] what the ‘something’ is.” The whole process harms many students as the social rewards for some are not available to others who “don’t make the grade (sic)” Should American public education have the function of sorting and separating students so that some may receive greater benefits than others, especially considering that the sorting and separating devices, educational standards and standardized testing, are so flawed not only in concept but in execution?
My answer is NO!!!!!
One final note with Wilson channeling Foucault and his concept of subjectivization:
“So the mark [grade/test score] becomes part of the story about yourself and with sufficient repetitions becomes true: true because those who know, those in authority, say it is true; true because the society in which you live legitimates this authority; true because your cultural habitus makes it difficult for you to perceive, conceive and integrate those aspects of your experience that contradict the story; true because in acting out your story, which now includes the mark and its meaning, the social truth that created it is confirmed; true because if your mark is high you are consistently rewarded, so that your voice becomes a voice of authority in the power-knowledge discourses that reproduce the structure that helped to produce you; true because if your mark is low your voice becomes muted and confirms your lower position in the social hierarchy; true finally because that success or failure confirms that mark that implicitly predicted the now self evident consequences. And so the circle is complete.”
In other words students “internalize” what those “marks” (grades/test scores) mean, and since the vast majority of the students have not developed the mental skills to counteract what the “authorities” say, they accept as “natural and normal” that “story/description” of them. Although paradoxical in a sense, the “I’m an “A” student” is almost as harmful as “I’m an ‘F’ student” in hindering students becoming independent, critical and free thinkers. And having independent, critical and free thinkers is a threat to the current socio-economic structure of society.
LikeLike
What is PROFOUNDLY troubling to me is that
GOVERNMENT, not scholarship defines “truth” by providing test questions – government answers as to what is “truth”, government validates the answers, not scholars, and our citizenry, who are now children, are judged on how well they can absorb government approved facts, not think for themselves, develop themselves to t heir highest potential as human beings etc etc etc.
THAT for me is the underlying fallacy of t his whole ugly mess. HOW can that be considered democratic.
Autocratic, indeed fascistic governments, THAT is what THEY do.
MUST we emulate that? Can we call ourselves a “land of the free and home of the brave” when we so quietly acquiesce to such a philosophy?
Yet it seems that the public does not grasp the significance of this.
LikeLike
“When the “reformers” start fighting with one another, ye shall know that real progress is being made.” — God
LikeLike
Her angry, hysteric response makes her look like a white western diplomat who is privileged to be an expert in non-western country turns into a ‘bully’ against his/her countrymen. It’s common tactic some apologist-like minded persons use to slam anyone who attempts to bring up the issue of social justice, such as racial discrimination and human rights abuse. It really reminds me of the article written by a white author who denounces westerners in poor defense of racial discrimination in foreign country.
http://www.gregoryclark.net/jt/page54/page54.html
LikeLike
AKA colonialism, as called out by Jitu Brown at NPE 2015 Chicago.
LikeLike
“Haycock jumps out of her daytime-TV chair, knocking it back as she rushes forward to get in Tucker’s face while declaring that she, “even a white girl,” can register what is Tucker’s obvious insult:”
Who says white girls can’t jump?
LikeLike
good one
LikeLike
This is interesting. John King appears at ed reform events, I guess in his capacity at the US Department of Education.
Does anyone from the Obama Administration attend events promoting public schools?
https://www.canvas.net/browse/excelined/courses/urgent-need-for-education-reform?utm_content=buffer180e1&utm_medium=social&utm_source=twitter.com&utm_campaign=buffer
LikeLike
We already know there is an achievement gap between wealthier families and poor families. An we know race and location correlate with income. So civil rights groups are using tests as a proxy to inequality to tell us what we already know, that inequality exists? What will be interesting is if test scores rise and inequality still exists. Hmmm. Could it be using tests as means to raise tests scores is a distraction to the real issues underlying inequality and poverty? Issues that politicians and business leaders want to ignore?
The entire testing craze will most certainly collapse under the growing ridiculousness of the whole approach. The tests may remain, but as yet another useless mandate teachers perform because people ignorant of teaching say it must be done. How many of us sit through meetings, fill out forms, type into computers in order to satisfy some requirement that serves no useful purpose?
LikeLike
Mathvale,
You hit the nail on the head. Reformsters prefer to talk about testing to distract attention from poverty, inequality, and racism
LikeLike
Peter Meyer
Here is the USDOE/NYSED/BOR/Cuomo reaction after low test scores in my/your school district got their attention:
Defunding
2% tax cap
Teacher/program cuts
Unmanageable class sizes
More testing
Excessive test score pressure
Constrained, test-prep curricula
Stigmatized by perpetual SINI/FOCUS status
Expensive, time wasting improvement plan requirements
LikeLike
Not sure about your claim regarding an absence of curricula??? Teachers have spent endless hours curriculum mapping on Rubicon Atlas, all readily available on the district website. In recent years (under the Regents Reform Agenda) teachers have been forced to constrain curricula, drastically reducing learning activities that should provide enrichment in favor of intensive ELA and math test prep. The external pressures to improve test scores have been enormous, permanently changing classroom learning environments for the worse. Your claim about “bad” pedagogy is baseless and an unnecessary insult to the hard working teachers in our district.
LikeLike
Sounds like NY Teacher is in my district, so he/she understandably has much to say about these issues. It’s a district worthy of study, including for the defensiveness of the last line. Despite spending almost $25,000 per student, the district has been routinely ranked near the bottom of a list of 83 districts in the region, a ranking made possible by the standardized tests required by NCLB. I have heard all the explanations for such poor performance — parents, poverty, etc — but no teacher should be insulted when the proffered explanations include bad curriculum or pedagogy.
LikeLike
PM
Your $25K per student figure is misleading and a bit disingenuous. You can’t simply divide the entire operating budget by the number of students and imply that this money is all channeled directly to the classroom. If that were true, just one class of 28 elementary students would be getting a $700,000 education.
Sorry to see that you want to define our school district and all its opportunities by the standardized tests scores in just two subject areas, in only half the grade levels.
“but no teacher should be insulted when the proffered explanations (for low test scores) include bad curriculum or pedagogy.”
This makes no sense. Our curriculum is aligned with all state standards and our teaching methods and practices are no different than any of the other 83 districts you mentioned. The opportunities for student growth and academic success are significant. The impressive achievements of our top students are a testament to this
fact.
LikeLike
Dear NY Teacher, I don’t want to use Diane’s blog to relive old fights, but it is interesting how familiar the non-dialogue is. Please read my earlier comments on the benefits of standardized tests before putting words in my mouth. And I’m not insinuating anything by stating that the district spends $25,000 per child. That’s what it spends. If you think more of that money should be going into the classroom, then you should lobby for that. But public school districts are charged with educating the children who walk through their door. And the district, by most standards, is not doing that very well — not for a majority of its kids. (And my argument about state mandates and all the top-down testing requirements, as already stated, is that a well-implemented content-rich core curriculum, vertically and horizontally aligned, will take care of most of these requirements.) Of course there are good things and good people — I wouldn’t have sent my son there for 12 years or worked 20 hours a week while on the school board for 5 years if I didn’t believe we had terrific resources — and the point I was making with my first comment here was that NCLB and standardized tests helped us see how many kids were not benefiting from those resources. And that insight did start us on an improvement effort that is working, however slowly.
LikeLike
NCLB/RTTT testing may have helped us identify our students who struggle in math and ELA, but the punitive nature of the federal testing agenda has done way more harm than good.
After NCLB testing spotlighted the problem, the most important solution should have been a true commitment to lowering class size and providing high quality pre-K programs. Instead of moving in that direction, standardized testing diverted important resources (time, money, energy) away from these goals. Further proof that the federal testing agenda was never about improving the quality of teaching and learning.
The federal testing program has been a lot like a a federal dental program that x-rays the teeth of all children, spots cavities, threatens and punishes the dentists and hygienists when their patients get cavities, does nothing to prevent or repair the cavities, and insists that they need bigger and more expensive x-ray machines.
LikeLike
I don’t disagree about possible solutions, but my experience of the teacher response to NCLB was anger and dismay on the part of teachers and an administration and BOE that enabled such a response by expressing their disenchantment sympathy with grudging implementation. That attitude, IMHO, delayed the improvement movement by a good five years. But my message then was the same as it is now: get a good curriculum, write it down, teach it, and test it — and you won’t have to worry about the NCLB testing requirements. I was preaching District agency! No law, no mandate prevented the district from having a great curriculum. What did prevent it was a school culture that did not understand what a coherent content-rich curriculum was. Just getting a BOE curriculum committee was a huge fight. Sure, “standardized testing diverted important resources (time, money, energy) away from these goals,” but whose decision was that? The district’s. Had district leaders embraced the powers they had and had the education community understood the power of a rich curriculum, they would not have diverted those resources. Why nobody did this is the $64k question.
LikeLike
Peter
I am in complete agreement regarding the need for a content rich curriculum. However all of the external pressures surrounding Common Core standards now drive instruction. The CC standards and their companion tests are completely devoid of content. Zero. Nada, Zilch. Big Fat Goose Egg. The Common Core standards are checklists of abstract, subjective skills, especially in ELA. The tests themselves do NOT test critical thinking skills, which rely on content knowledge, either. The claim that these standards and tests will ensure college and career readiness is a pure snake oil sales job.
The content rich curriculum that would best serve our students will never be implemented as long as Common Core standards and the test-and-punish attachments remain in place. Please don’t blame our district for ignoring content knowledge because the current federal testing regime requires that we focus on abstract,skills that are un-teachable; abstract skills that completely fail to excite and inspire any normal child.
LikeLike
I understand the outside pressures, but they are misguided. The CCSS state very clearly that the standards are not sufficient; that a curriculum must be written. So, if there is no curriculum being written and used, it is in violation of the express suggestions of the CC.
LikeLike
Okay, Kati Haycock is a tool. We’ve known that for while. And she doesn’t build many of her arguments around facts; that much is well-known too.
But in her post, Mercedes Schneider seems to be offering back-door support for Marc Tucker as much as she.gives a frontal slap to Haycock.
And that’s worth taking a look at.
Tucker has criticized the current regime of testing, to his credit. He says he wants to replace the current system with one that that has fewer tests with higher stakes. But, as Yong Zhao pointed out:
“Tucker’s new plan includes three high stakes tests for all students at 4th, 8th, and 10th grades, but expanded to more subjects: math, English, and “subjects like history, literature, science, social studies, music, and the arts.” Additionally, Tucker wants a first grade ‘Diagnostic Test’ and “every other off year, the state would administer tests in English and mathematics beginning in grade 2, and, starting in middle school, in science too, on a sampling basis.”
So, in an effort to “fix” the system, Tucker would add “more tests, more high stakes tests, and more standardized tests.”
This cannot reasonably be called an improvement.
Too, the rationale for Tucker’s fix is that national economic growth and prosperity are dependent on student test scores. And that’s just plain wrong.
There’s a reason that the U.S. Chamber of Commerce and the Business Roundtable are all-in on the Common Core. And there’s a reason that they have an ally in Marc Tucker, who gets funding from the likes of the Gates and Broad Foundations along with a host of Walton organizations.
Kati Haycock is usually in error in what she supports as education “reform.”
So is Marc Tucker.
LikeLike