Marc Tucker posted a fascinating dialogue with two testing experts, Howard Everson and Robert Linn.
Here are some of the salient points.
MT: Is this country getting ready to make a profound mistake? We use grade-by-grade testing in grades 3-8 but no other country is doing it this way for accountability; instead they test 2 or 3 times in a students’ career. If the United States did it that way, we could afford some of the best tests in the world without spending any more money.
BL: Raising the stakes for our test-based accountability systems so that there will be consequences for individual teachers will make matters even worse. Cheating scandals will blossom. I think this annual testing is unnecessary and is a big part of the problem. What we should be doing is testing at two key points along the way in grades K-8, and then in high school using end-of-course tests.
HE: I am in the same place as Bob. The multiple-choice paradigm first used in WWI and eventually used to satisfy the NCLB requirements has proven to be quite brittle, especially when applied in every grade 3-8 and used to make growth assumptions. The quick and widespread adoption of multiple-choice testing was in hindsight a big mistake for this country, but—now — states will tell you it is all they can afford.
Bob Linn points out that the increased reliance on external tests reflects a fundamental distrust of teachers. Our nation relies on these tests because teachers can’t be trusted to test their own students. The conundrum is that the reliance on standardized tests demoralizes and deskills teachers and reduces the prestige of the teaching profession
BL: One big difference between the United States and other countries is the prestige and trust in teachers, which is very low in this country and tends to be quite high in the top performers. This has led to the development of accountability systems that use external measures to see if schools and individual teachers are doing a good job. This has morphed into the next level: evaluating individual teachers. Unless we can find a way to increase the prestige of teachers and public confidence in them, it will be hard to move too far away from using testing for these purposes.
I am not a testing expert. I am a historian. I have studied the history of testing (see Left Back), and I served for seven years on the National Assessment Governing Board. One thing I know: the testing industry is the greatest beneficiary of the testing mandates of No Child Left Behind and the Race to the Top. The testing industry has lobbyists who look out for the interests of the industry. They work on Capitol Hill and in the state capitols.
Can they be stopped? Can we bring the best interests of children to the fore, to replace the best interests of the testing industry?
Yes.
Opt out.
Do not allow them to test your child.
Show your disdain for their flawed product.
Do not allow them to use your children as data points.
Diane Ravitch's blog 
He’s a historian? Then why again, implement Outcome based ed (now called competency based ed) the failed fad upon the nation.?
LikeLike
No. Diane is a historian. The italics are the two “testing experts” and the regular print is Diane.
LikeLike
If testing every year is bad (which I agree with), then why is testing at all good? How are these tests going to be better if they’re only given a couple times in a student’s career. They say we’ll be able to afford “better” tests – what would that look like?
LikeLike
Remember, Dienne, they’re the “experts” and we novices should bow down before their “expertise”.
LikeLike
I don’t know that I think testing in of itself is the largest problem, but what is tied to the testing. Currently it is used as a measure of a teacher’s (or students) worth when is should be used as a single data point for assessing how students are learning compared to their peers in other districts, states, countries, etc. Test results should not be used for concrete evaluations of any particular individual, but only for a general sense to identify any trends or inconsistencies in the data. I would be okay with several key exams through the academic life of the student, but they should remain fairly anonymous and the outcome of the exam should not be tied to teacher performance, student advancement, or funding.
LikeLike
Derrick,
Not seen you comment yet, welcome!
You stated “I don’t know that I think testing in of itself is the largest problem, but what is tied to the testing. Currently it is used as a measure of a teacher’s (or students) worth”
Beg to differ but yes standardized testing is the biggest problem, with the resultant using of the results as a supposed measure of a teacher’s or student’s worth being totally UNETHICAL.
I invite you to hop on the Quixotic Quest Badwagon in helping me rid the world of these nefarious educational malpractices that are educational standards, standardized testing and the “grading” of students. You can get your free ticket by reading and understanding why those malpractices are completely invalid in Noel Wilson’s “Educational Standards and the Problem of Error” found at: http://epaa.asu.edu/ojs/article/view/577/700
Brief outline of Wilson’s “Educational Standards and the Problem of Error” and some comments of mine. (updated 6/24/13 per Wilson email)
1. A quality cannot be quantified. Quantity is a sub-category of quality. It is illogical to judge/assess a whole category by only a part (sub-category) of the whole. The assessment is, by definition, lacking in the sense that “assessments are always of multidimensional qualities. To quantify them as one dimensional quantities (numbers or grades) is to perpetuate a fundamental logical error” (per Wilson). The teaching and learning process falls in the logical realm of aesthetics/qualities of human interactions. In attempting to quantify educational standards and standardized testing we are lacking much information about said interactions.
2. A major epistemological mistake is that we attach, with great importance, the “score” of the student, not only onto the student but also, by extension, the teacher, school and district. Any description of a testing event is only a description of an interaction, that of the student and the testing device at a given time and place. The only correct logical thing that we can attempt to do is to describe that interaction (how accurately or not is a whole other story). That description cannot, by logical thought, be “assigned/attached” to the student as it cannot be a description of the student but the interaction. And this error is probably one of the most egregious “errors” that occur with standardized testing (and even the “grading” of students by a teacher).
3. Wilson identifies four “frames of reference” each with distinct assumptions (epistemological basis) about the assessment process from which the “assessor” views the interactions of the teaching and learning process: the Judge (think college professor who “knows” the students capabilities and grades them accordingly), the General Frame-think standardized testing that claims to have a “scientific” basis, the Specific Frame-think of learning by objective like computer based learning, getting a correct answer before moving on to the next screen, and the Responsive Frame-think of an apprenticeship in a trade or a medical residency program where the learner interacts with the “teacher” with constant feedback. Each category has its own sources of error and more error in the process is caused when the assessor confuses and conflates the categories.
4. Wilson elucidates the notion of “error”: “Error is predicated on a notion of perfection; to allocate error is to imply what is without error; to know error it is necessary to determine what is true. And what is true is determined by what we define as true, theoretically by the assumptions of our epistemology, practically by the events and non-events, the discourses and silences, the world of surfaces and their interactions and interpretations; in short, the practices that permeate the field. . . Error is the uncertainty dimension of the statement; error is the band within which chaos reigns, in which anything can happen. Error comprises all of those eventful circumstances which make the assessment statement less than perfectly precise, the measure less than perfectly accurate, the rank order less than perfectly stable, the standard and its measurement less than absolute, and the communication of its truth less than impeccable.”
In other word all the logical errors involved in the process render any conclusions invalid.
5. The test makers/psychometricians, through all sorts of mathematical machinations attempt to “prove” that these tests (based on standards) are valid-errorless or supposedly at least with minimal error [they aren’t]. Wilson turns the concept of validity on its head and focuses on just how invalid the machinations and the test and results are. He is an advocate for the test taker not the test maker. In doing so he identifies thirteen sources of “error”, any one of which renders the test making/giving/disseminating of results invalid. As a basic logical premise is that once something is shown to be invalid it is just that, invalid, and no amount of “fudging” by the psychometricians/test makers can alleviate that invalidity.
6. Having shown the invalidity, and therefore the unreliability, of the whole process Wilson concludes, rightly so, that any result/information gleaned from the process is “vain and illusory”. In other words start with an invalidity, end with an invalidity (except by sheer chance every once in a while, like a blind and anosmic squirrel who finds the occasional acorn, a result may be “true”) or to put in more mundane terms crap in-crap out.
7. And so what does this all mean? I’ll let Wilson have the second to last word: “So what does a test measure in our world? It measures what the person with the power to pay for the test says it measures. And the person who sets the test will name the test what the person who pays for the test wants the test to be named.”
In other words it measures “’something’ and we can specify some of the ‘errors’ in that ‘something’ but still don’t know [precisely] what the ‘something’ is.” The whole process harms many students as the social rewards for some are not available to others who “don’t make the grade (sic)” Should American public education have the function of sorting and separating students so that some may receive greater benefits than others, especially considering that the sorting and separating devices, educational standards and standardized testing, are so flawed not only in concept but in execution?
My answer is NO!!!!!
One final note with Wilson channeling Foucault and his concept of subjectivization:
“So the mark [grade/test score] becomes part of the story about yourself and with sufficient repetitions becomes true: true because those who know, those in authority, say it is true; true because the society in which you live legitimates this authority; true because your cultural habitus makes it difficult for you to perceive, conceive and integrate those aspects of your experience that contradict the story; true because in acting out your story, which now includes the mark and its meaning, the social truth that created it is confirmed; true because if your mark is high you are consistently rewarded, so that your voice becomes a voice of authority in the power-knowledge discourses that reproduce the structure that helped to produce you; true because if your mark is low your voice becomes muted and confirms your lower position in the social hierarchy; true finally because that success or failure confirms that mark that implicitly predicted the now self evident consequences. And so the circle is complete.”
In other words students “internalize” what those “marks” (grades/test scores) mean, and since the vast majority of the students have not developed the mental skills to counteract what the “authorities” say, they accept as “natural and normal” that “story/description” of them. Although paradoxical in a sense, the “I’m an “A” student” is almost as harmful as “I’m an ‘F’ student” in hindering students becoming independent, critical and free thinkers. And having independent, critical and free thinkers is a threat to the current socio-economic structure of society.
LikeLike
I was by no mean advocating for standardized testing, and in fact, I am certainly opposed to using them to evaluate children’s or teacher’s performance. My point was that a test in of itself is not necessarily harmful, but what is done with the data is the problem. The data could be useful to educational researchers as more of a meta-analysis tool to identify anomalies in data trends, but not useful as an evaluation of student and teacher performance. By not linking a test to performance, there would be no need to prepare for the test and it could be a more accurate representation of what students have learned (assuming a well designed test which may be a big assumption). Every year we can see rankings in various subjects by districts, state, country, etc. This data needs to come from somewhere, and I believe it is useful as long as it is not individualized. So I am not opposed to several tests throughout the academic career of a student, as long as that data is used strictly for the purpose of building data sets and not evaluating students. We had several tests like this, when I was younger, well before the age of standardization and accountability. We never prepared to take these tests, and they had no bearing on our (or our teachers’) academic or professional careers.
LikeLike
I agree with this post. I believe the privatizers will go to great expense to disenfranchise teacher education programs and teachers themselves. It isn’t about the kids. It is about the testing industry.
We have several camps of objections, I think. One is against the CC. Another is against the tests. Another is against both. Reasons are myriad. Support comes from many directions. I think there are a group of parents who are obsessed with controlling their children’s education from birth. They want to fire teachers and treat them as nannies. They want to command the teachers’ thoughts and curricula for their individual child. So they buy into privatization . They aren’t interested in anyone else’s opinion. They know it all. That is the tip of the iceberg. But they are part of the reason that this insanity has been able to move forward. Others are just lemmings or disgruntled about schools from the time the were in first grade. It is very politically motivated if you look at the groups that support this movement.
LikeLike
Dear Diane,
I know you have lots more on your plate than to answer specific questions for concerned moms, but if you have a minute could you tell me if you believe I should opt my 6th grade son out of the CogAT? I just got a notice from his school that they will be giving these next week.
I’ve done some research to find out what it’s for (gifted placement?). He’s already in the “gifted program” which is a very sad, flaccid thing lately… and I’m not really worried that he won’t do well (as a matter of fact I think he does well on reasoning tests), but I’m concerned that these could be used somehow against him… or just for data-mining, etc.
Do you have any thoughts on the CogAT for a mom trying to do the right thing?
Thanks so much for your books and for what you are doing.
LikeLike
I don’t know that particular test. As a general rule, it is a good idea to remove your child from standardized testing because they are not used for diagnostic purposes, as they should be, but just to hand out carrots and sticks. They are misused. It is a waste of time and money.
LikeLike
As a hearing therapist, I depend upon (specialized) standardized tests to confirm or refute my ‘gut’ appraisal of a student’s functioning and to inform about levels of specialized support needed. Test performance strengths and weaknesses also determine the focus of the next years therapy goals on the IEP. Every once in awhile, test results are contrary to the student’s functioning…but not often. These test results also have prompted me to request further student assessment for ‘giftedness’, speech, and academic support…and have even prompted me to request ‘hearing aid evaluations’ if I suspect the current device is not adequate for the student’s need. But it is not the final single ‘score’ that is the treasure….I study the error patterns and clusters; look for the point at which fatigue sets in, etc. That is where the ‘juicy’ information lies…not in the final score. For diagnostic purposes, the tests can be wonderful, like a broad overview of abilities and a window in to the details. The tests also provide me with valuable feedback to ‘me’ regarding my own professional judgments and abilities…and have even prompted me to seek further reading, training and coursework in areas I need more skill. Tests, interaction, observation and collaboration…a very nice package of valuable student info. Wouldn’t write a report without it!
LikeLike
According to Duncan, we must test every year to hold students and teachers accountable. He never discusses how mass testing funnels profits to Pearson’s shareholders and undisclosed corporate interests. Duncan listens to lobbyists affiliated with the testing industry, not parents, teachers or the legislators in California and Texas.
http://jasonstanford.org/2013/09/when-california-and-texas-agree/
LikeLike
“Bob Linn points out that the increased reliance on external tests reflects a fundamental distrust of teachers. Our nation relies on these tests because teachers can’t be trusted to test their own students.”
I don’t know if it necessarily reflects a “fundamental distrust” so much as an over-reliance on a pseudo-scientific means of “assessing” students that people have been led to believe are unbiased (when in fact all assessing has “bias” built into it. Standardized tests have claimed the mantel of scientificity, of unbiasedness, of value freeness, etc. . . when in reality that claim is false.
LikeLike
Duane Swacker: if I may, let me put my own spin on this.
The leading lights of the charterite/privatize movement don’t necessarily all have a “fundamental distrust” of teachers—because that would imply that they know what teachers should do, have done, and are doing. Teachers are simply in the way of their pursuit of EduExcellence aka $tudent $ucce$$.
Aside from a few self-deluded numbers/stats folks [including those who literally are not psychometricians so are out of their field of expertise in any case], the vast majority of those Most Cagebusting Leaders of the Civil Rights Movement of the Twenty First Century don’t have a clue about standardized tests. Former President George W. Bush Jr. would find plenty of current co-signers to his statement in 2001 that “A reading comprehension test is a reading comprehension test. And a math test in the fourth grade—there’s not many ways you can foul up a test… It’s pretty easy to ‘norm’ the results” [cited in Daniel Koretz, MEASURING UP: WHAT EDUCATIONAL TESTING REALLY TELLS US, 2009, p. 7). *Hint: this statement is not just wrong, it is bizarrely uninformed.*
One of the most appealing features of high-stakes standardized testing for at least 98% [thank you, Bill Gates!] of edufrauds everywhere is that the resulting scores seem to work like conscience-relieving automatons, furiously clubbing school staffs and students and parents with an admirable [?] mechanical precision and passionless fury. The pseudo-certainty that comes of labeling, sorting and ranking based on allegedly ‘objective mathematical formulae’—and the resulting [infrequent] rewards and [numerous] punishments meted out to millions and millions of people—lets them feel that they are off the hook. Accountable, maybe, but responsible—never. That is why they can do their ‘strut and puff’ walks in public while destroying public education, swelled up in photo-ops to appear like demigods, all the while expressing their genuine (Rheeally!) sorrow for the ‘bad’ teachers and student ‘non-strivers’ who just didn’t measure up. **Of course, their children and the schools they go to are exempted from the Hazing Rituals of the High Church of Testolatry.**
Let me end on a more personal note. As you already know, I periodically get besieged—you know from firsthand experience too—with requests from aggrieved groups to correct unfair stereotypes that appear on this blog.
The League of Extraordinary Frauds (foundation inspired by Charles Ponzi) boasts [?] an impressive roster of, er, “the best and the brightest” including the brains behind the Gulf of Tonkin incident and WMD in Iraq, among many many others (their standards are so strict that Bernie Madoff doesn’t even qualify for a small plaque on their wall).
I have been asked to quit using the felicitous term that Linda came up with: “edufraud.” For reasons of both professional and personal honor [?] they feel that the current crop of self-styled “education reformers” has so cheapened the term “fraud” that they may have to rename their fraternal [?] organization.
However, to show respect to the owner of this blog, I refuse to comply. As Gertrude Stein might have said, an “edufraud is an edufraud is an edufraud is an edufraud” followed up by “There is no there there.” [She actually wrote the second]
I am so sure that you agree with me that I have asked them to stop spamming my email inbox and yours with this demand.
Los sanos unidos jamás serán vencidos/the healthy/sane will never be defeated.
🙂
LikeLike
As a teacher who is now awaiting the decision of my former district regarding my termination due to low test scores as part of my teacher evaluation, I couldn’t agree more with the comments regarding the demoralization of teachers who teach in the lowest performing public schools. My “evaluators” who would show up for 10 minutes in my class were constantly criticizing my efforts when they knew little about my kids, or where they were academically and socially/emotionally. There were to be no excuses for poor performance…I felt more like I was making widgets than educating children.
Do policy makers and the public really believe that all students are created equal and that there can really be “one size fits all” solutions that only address academic goals? It is so ridiculously naive! Do they have any understanding that having a class of 27 kids from extremely dysfunctional families with little support feels like teaching 100 students at once, especially when at least half of them read below grade level and many have serious problems with attention (often caused by everything from poor nutrition, no prenatal care, violence in the home, bullying in the school, etc., etc.)? How many have spent any time in these schools or talked to teachers on the front line to see how complex an equation it is to educate children who have such enormously complex lives?
Diane is “right on the money” when she suggests that limiting class size for these types of children is a great first step, but it is only the beginning…as teachers, if we can’t bring the joy of learning to these very reluctant children by meeting them “where they are” emotionally and as learners, then we lose them…what evaluators saw as my “low expectations” for my students, I saw as giving them confidence in their nascent abilities.
One of my many strategies was to occasionally use motivating films like those about real people who changed attitudes about the importance of education like “Coach Carter” or one about Ben Carson (who rose from poverty to become a world-renowned surgeon) in order to inspire my students. These films were also accessible to all students regardless of reading level. We would discuss and write about these films, to reinforce those skills, but I was told that I was “wasting” instructional time, and these efforts were not rigorous. Don’t they understand that rigor is relative to who is in the classroom, and that what is rigorous for an 8th grader reading on grade level is overwhelming to the 8th graders who may be slow processors or ELL students? The constant testing just serves to reinforce their feelings of inadequacy.
I believe that there are a large number of policy makers who have no clue that we must “sell” the idea of the importance of education to at-risk children because they have little experience with the concept. Finland has it right in their approach and so did Maria Montessori in her work with children in the slums of Rome…instill a love of learning because it is fun, because it brings confidence, and build from there…then informal and formal assessment can follow. Charter schools are jumping the gun, and in some cases trying to make automatons out of children with their methods.
It is easy for the uninformed to just blame the “lazy, inept teachers” for the problems facing our students, but I fear for our country if we don’t stop this insanity. It’s like blaming the dentist for the patient who develops a cavity! Personally, I am done with the battle and the anxiety that was ruining my physical and mental health…although I will always love the students whose lives I touched (and who touched mine) at 58 years old, I will pursue a career in real estate.
LikeLike
Your very eloquent post nearly brought me to tears (and righteous rage). We keep terminating teachers like you who obviously care about and understand children, particularly the more challenging and needy ones, and then we wonder what’s wrong with schools/kids these days, why are kids so “unmotivated”?
I wish you well and I’m pulling for you not to be terminated. But if you are, I would encourage you to consider blogging or writing a book or something to get the word out about your experience. If this post is an example, you are a powerful writer.
LikeLike
If termination is a direct result of test scores, I would strongly encourage her to get a lawyer!
LikeLike
I sent a message to one of our school board members giving him info on parental rights for opting out of these tests and he replied that opting out our children will hurt their scores when it comes time for SAT and ACT tests. Would you please reply to this?
LikeLike
If everyone opts out, then the scores won’t matter. Don’t let the Machine rule your life or the life of your child. It feeds on data. Deny it the food it needs. Bring the Machine to a halt. Organize other parents. If everyone does it, there will be no consequences for anyone.
LikeLike
Thank you so much for your very kind words, Dienne…unfortunately, there are so many more teachers in this country with very similar experiences…and the saddest thing of all is what the “Emperor’s New Clothes” policies are doing to our most “at-risk” children…and really, really frightening…
LikeLike
. . . and make it a class action civil suit.
LikeLike
Yes, legal action is being considered, but when the union members voted for the contract that included this teacher evaluation formula three years ago in an effort to get Race to the Top money, we agreed to live with the test score tie-in, so there is little legal protection. It is a complicated issue, because teacher ratings are now conveniently being used by some principals as a way to get rid of teachers who make them uncomfortable…test scores aren’t the only measure for the evaluation, but the principals can be very subjective in their ratings, partly because some principals don’t appear to entirely understand the rubric for these ratings. We only got a handbook about the evaluation system two years after it was in place because they had to rush to get it implemented so they would qualify for that federal money.
It is sort of like asking teachers to win a board game without knowing all of the rules…
LikeLike
A class action suit based on the scientific inaccuracy of the tests might well be called for.
LikeLike
Years ago, I was working away in my office when I heard a series of loud thumps. I went out to the lobby to investigate. Along one end of the lobby was a bunch of plate glass windows overlooking a deck and, beyond that, a marsh. A very large seagull was standing in front of the plate glass window and banging his beak against his own reflection in the glass. The seagull would bang the glass, stagger back, and bang again. I went out and shooed him away before he knocked himself completely silly.
It’s the height of stupidity to be doing something that obviously isn’t working and to continue doing that. It’s even more crazy to be doing something that isn’t working and to ratchet it up, to do a LOT MORE of that.
But that’s exactly what we are doing with the new standards and the new exams. After more than a decade of having turned our schools into test prep centers because of NCLB, ,after more than a decade of the narrowing and distortion of pedagogy and curricula that resulted from that, after the UTTER FAILURE of the NCLB testing machine, our oh-so-brilliant leaders have, seagull-like, decided that we need to continue doing what we’ve been doing. And worse yet, they have decided that we need to do A LOT MORE OF THAT.
Dumb. Dumb. Dumb.
Birdbrained.
LikeLike
By “A LOT MORE THAN THAT” I mean, of course, that they have decided to ratchet up the stakes of the exams, while the plutocrats backing the standards have bankrolled trainings in districts around the country based on the Scylla and Charybdis of the new standards [sic] on the one hand and the tests on the other.
LikeLike
Bill Gates decided that the way to improve U.S. education was to improve teacher quality. He mistakenly thinks that
the way to measure teacher quality is via these high-stakes exam and that
the way to improve teacher quality is to remove teachers based on these exams.
In a recent interview, he stated his goal of having the teachers in the bottom three quartiles be of the quality of the teachers now in the top quartile (as measured by exam results).
Let me translate that: get rid of 3/4ths of the teachers in the United States.
I am all for improving teacher quality. But this is not how to go about it. Sadly, BTW, Gates has decided teachers’ getting advanced degrees doesn’t matter. That’s because the stats show that when teachers get Masters degrees in education, their students’ test scores don’t improve.
The problem, of course, is that the tests are lousy instruments for making these kinds of evaluations.
It’s astonishing that people could allow their lousy metrics to lead them to believe that the educational attainments of teachers don’t matter. But there it is: That’s what Bill believes, and as we all knows, ALL THAT MATTERS IS WHAT BILL THINKS. He’s they guy with the checkbook.
LikeLike
cxs:
“the way to measure teacher quality is via these high-stakes EXAMS”
and
“as we all know,”
of course.
I do wish one could edit these posts!
LikeLike
Yeah, I had a 3.97/4.0 GPA in my Masters’ program 10 years ago and I am dual certified in middle school English and Social Studies, and worked in marketing research in Japan and the US before becoming a teacher…I guess it doesn’t make sense to have someone like me in the classroom…
LikeLike
“Sadly, BTW, Gates has decided teachers’ getting advanced degrees doesn’t matter.”
What would Billy the Goates know about a post grad degree???? The closest he’ll come to one is when and if he buys one.
“That’s because the stats show that when teachers get Masters degrees in education, their students’ test scores don’t improve.”
I hope that’s because by getting an advanced degree they realize that student tests scores are the very last thing that should be focused on. Better yet why waste any energy whatsoever in even thinking about, much less trying to raise them.
LikeLike
Diane, Do you still work with Gordon Commission. How do their conclusions regarding assessments jibe with Tuckers comments?
LikeLike
Hi Diane – Not sure if you’ve already seen this or whether it is of interest.
Suzanne Harris, avid Ravitch blog reader & supporter!
LikeLike