Quinn Mulholland of the Harvard Political Review examined the issues surrounding annual mandated testing, interviewed leading figures on both sides, and concluded that the exams are overkill. They cost too much, they narrow the curriculum, they take too many hours, they distort the purpose of education.
Mulholland concludes:
Given all of these problems with standardized testing, it seems that the civil rights issue is too much testing, not too little. Instead of forcing low-income schools to spend millions of dollars and countless hours of class time preparing for and administering standardized tests that only serve to prove, oftentimes inaccurately, what we already know about the achievement gap, we should use those resources to expand programs in the arts and humanities, to provide incentive pay to attract teachers to areas where they are needed most, and to decrease class sizes, all things that could actually make a difference for disadvantaged students.
This is not to say that America’s accountability system should be completely dismantled. Politicians and schools can de-emphasize testing while still ensuring high achievement. Student and teacher evaluations can take multiple measures of performance into account. The amount of standardized tests students have to take can be drastically reduced. The fewer standardized tests that students do take can incorporate more open-ended questions that force students to think critically and outside the box
Thirteen years after NCLB’s mandates were first set into place, the rhetoric used by politicians and pundits is sounding more and more like that which the same politicians and pundits used to endorse NCLB. Congress would be ill advised to try to use high-stakes test-based accountability to narrow the achievement gap and expect a different result than the aftermath of the 2002 law. It is time to acknowledge that putting an enormous amount of weight on standardized test scores does not work, and to move on to other solutions.
Regardless of the outcome of the current debate, grassroots activists like [Jeanette] Deutermann will continue to fight against harmful test-based accountability systems like New York’s. “This is an epidemic,” she said. “It’s happening everywhere, with all sorts of kids, from the smartest kids to the kids that struggle the most, from Republicans to Democrats, from kids in low-income districts to kids in high-performing districts. It doesn’t matter where you are, the stories are exactly the same.”
“We may be passive when it comes to all the other things [corporate reformers] have interjected themselves into,” Deutermann warned, “but when you mess with our kids, that’s when the claws come out.”
Diane Ravitch's blog 
It is time to acknowledge that putting an enormous amount of weight on standardized test scores does not work, and to move on to other solutions.”
Move on to other solutions, but ONLY if there is evidence that they work.
It’s not OK to simply keep trying “solutions” in the absence of evidence that they work.
Imagine if engineers took such an unscientific (indeed idiotic) approach to building bridges.
The results would be entirely predictable — as they were for standardized testing and VAM.
LikeLike
I’d suggest they worry less about the “solutions” and more about accurately determining the (often alleged) problems. If the educational system isn’t actually failing (except when it’s being intentionally failed), perhaps there is no need for a “solution”. Unfortunately, none of them will ever be able to acknowledge the systems that are failing because it would make too many Harvard types too uncomfortable.
LikeLike
Just wonder if Harvard Graduate School of Education faculty ever bother to read the Harvard Political Review, especially the faculty who depend on the continuing flow of ready-to-use test scores for their research, their huge grants, their endowed professorships, their research assistants, and their national recognition as “experts” in education. Among others, all ECONOMISTS, are Raj Chetty, Ronald Ferguson, Thomas Kane.
These scholars have directly and indirectly spread the idea that policy in education should be shaped by number crunching and a pathetically reductive view of student achievement and teacher “effectiveness.” They have exaggerated the importance of scores on standardized academic tests, scores on student surveys (Ferguson designed the Tripod student survey), scores on Charlotte Danielson’s rubrics as measures of “effective teaching” (used to score videos of teaching instead of getting into real classrooms).
Sorry to be ranting, sort of.
Quinn MulHolland writes well enough. However the reification of test scores in educational policy owes much to faculty in the Graduate School Of Education at Harvard whose work so often depends on the reification of test scores, their pitifully truncated view of what counts in education, and their studied disregard for the damage being done to public education by exaggerating the importance of test scores.
LikeLike
Harvard’s reputation, IMO, heading south.
Adding to Laura’s listing:
The papers of Harvard economists Rogoff and Rhinehardt, while not focused on education, were flawed, as exposed by Paul Krugman, and extremely detrimental in impact.
A couple of days ago, Harvard-Smithsonian announced a change in policy, citing the impetus, which was a researcher who didn’t disclose funding from the fossil fuel industry. (Koch is on the Smithsonian Board).
2014 Harvard School of Government interns were placed with the Heritage Foundation and DFER, while public education advocacy organizations had none.
Two Harvard professors serve on a charter school board, alongside a controversial, high-visibility, free market economist.
Economics professor, Edelman’s well-publicized tirade against a small mom and pop restaurant.
An anomaly on campus, Prof. Lawrence Lessig.
LikeLike
Over and over and over these scholarly researches come up with the same result.
WOULD that the power structure which rules our country now would take heed. Paraphrasing, the first casualty of education is “truth”.
LikeLike
“Take Heed — and chuck the rest”
Our leaders only heed
The data that they need
The rest they simply dis
Cuz ignorance is bli$$
LikeLike
Perfect poem. Thanks.
LikeLike
I think the fact that we are still studying this is misdirected. There was sufficient evidence years ago to see that standardized testing does more harm than good.
So we need to pivot because this makes it seem like we are still unsure. If we only had 50 or 100 more studies, then politicians would listen, right? Wrong.
They aren’t listening.
LikeLike
Academia and “research” say lots of different things, often using the very same data. A big part of the problem is this “gold standard” thinking, as if teaching and learning were synonymous with treating patients or building a house on the right kind of foundation. Teaching is so much more of an art, experience, relationship that I think it makes even some of its own practitioners nervous. Many of us want it to be scientific, but the truth is, it’s not. It’s mostly trial and error, and what works with one group often bombs with another. All students and all teachers know that good teaching involves little science but much empathy, sensitivity, and willingness to change.
LikeLike
Wasn’t Harvard instrumental in development of CCSS and the attached high-stakes testing?
LikeLike
Becky,
I don’t think Harvard ever was involved in development of CCSS. The prime mover was David Coleman, who had no connection to Harvard.
LikeLike
“. . . and concluded that the exams are overkill. They cost too much, they narrow the curriculum, they take too many hours, they distort the purpose of education.
Mulholland concludes:
Given all of these problems with standardized testing. . . ”
And from Jake Jacobs:
“I think the fact that we are still studying this is misdirected. There was sufficient evidence years ago to see that standardized testing does more harm than good.”
Yes, Jake quite correct. As a matter of fact “given all these problems with standardized testing” and with the flip side of that educational malpractice coin, educational standards as identified by Noel Wilson “studying this is [not only] misdirected” but an astonishing waste of time, effort and scarce educational monies. To understand “all these problems” and harms foisted upon students one should read and understand Wilson’s seminal, never refuted nor rebutted 1997 dissertation (the most important piece of educational writing in the last half-century) “Educational Standards and the Problem of Error” found at: http://epaa.asu.edu/ojs/article/view/577/700
Brief outline of Wilson’s “Educational Standards and the Problem of Error” and some comments of mine.
1. A description of a quality can only be partially quantified. Quantity is almost always a very small aspect of quality. It is illogical to judge/assess a whole category only by a part of the whole. The assessment is, by definition, lacking in the sense that “assessments are always of multidimensional qualities. To quantify them as unidimensional quantities (numbers or grades) is to perpetuate a fundamental logical error” (per Wilson). The teaching and learning process falls in the logical realm of aesthetics/qualities of human interactions. In attempting to quantify educational standards and standardized testing the descriptive information about said interactions is inadequate, insufficient and inferior to the point of invalidity and unacceptability.
2. A major epistemological mistake is that we attach, with great importance, the “score” of the student, not only onto the student but also, by extension, the teacher, school and district. Any description of a testing event is only a description of an interaction, that of the student and the testing device at a given time and place. The only correct logical thing that we can attempt to do is to describe that interaction (how accurately or not is a whole other story). That description cannot, by logical thought, be “assigned/attached” to the student as it cannot be a description of the student but the interaction. And this error is probably one of the most egregious “errors” that occur with standardized testing (and even the “grading” of students by a teacher).
3. Wilson identifies four “frames of reference” each with distinct assumptions (epistemological basis) about the assessment process from which the “assessor” views the interactions of the teaching and learning process: the Judge (think college professor who “knows” the students capabilities and grades them accordingly), the General Frame-think standardized testing that claims to have a “scientific” basis, the Specific Frame-think of learning by objective like computer based learning, getting a correct answer before moving on to the next screen, and the Responsive Frame-think of an apprenticeship in a trade or a medical residency program where the learner interacts with the “teacher” with constant feedback. Each category has its own sources of error and more error in the process is caused when the assessor confuses and conflates the categories.
4. Wilson elucidates the notion of “error”: “Error is predicated on a notion of perfection; to allocate error is to imply what is without error; to know error it is necessary to determine what is true. And what is true is determined by what we define as true, theoretically by the assumptions of our epistemology, practically by the events and non-events, the discourses and silences, the world of surfaces and their interactions and interpretations; in short, the practices that permeate the field. . . Error is the uncertainty dimension of the statement; error is the band within which chaos reigns, in which anything can happen. Error comprises all of those eventful circumstances which make the assessment statement less than perfectly precise, the measure less than perfectly accurate, the rank order less than perfectly stable, the standard and its measurement less than absolute, and the communication of its truth less than impeccable.”
In other word all the logical errors involved in the process render any conclusions invalid.
5. The test makers/psychometricians, through all sorts of mathematical machinations attempt to “prove” that these tests (based on standards) are valid-errorless or supposedly at least with minimal error [they aren’t]. Wilson turns the concept of validity on its head and focuses on just how invalid the machinations and the test and results are. He is an advocate for the test taker not the test maker. In doing so he identifies thirteen sources of “error”, any one of which renders the test making/giving/disseminating of results invalid. And a basic logical premise is that once something is shown to be invalid it is just that, invalid, and no amount of “fudging” by the psychometricians/test makers can alleviate that invalidity.
6. Having shown the invalidity, and therefore the unreliability, of the whole process Wilson concludes, rightly so, that any result/information gleaned from the process is “vain and illusory”. In other words start with an invalidity, end with an invalidity (except by sheer chance every once in a while, like a blind and anosmic squirrel who finds the occasional acorn, a result may be “true”) or to put in more mundane terms crap in-crap out.
7. And so what does this all mean? I’ll let Wilson have the second to last word: “So what does a test measure in our world? It measures what the person with the power to pay for the test says it measures. And the person who sets the test will name the test what the person who pays for the test wants the test to be named.”
In other words it attempts to measure “’something’ and we can specify some of the ‘errors’ in that ‘something’ but still don’t know [precisely] what the ‘something’ is.” The whole process harms many students as the social rewards for some are not available to others who “don’t make the grade (sic)” Should American public education have the function of sorting and separating students so that some may receive greater benefits than others, especially considering that the sorting and separating devices, educational standards and standardized testing, are so flawed not only in concept but in execution?
My answer is NO!!!!!
One final note with Wilson channeling Foucault and his concept of subjectivization:
“So the mark [grade/test score] becomes part of the story about yourself and with sufficient repetitions becomes true: true because those who know, those in authority, say it is true; true because the society in which you live legitimates this authority; true because your cultural habitus makes it difficult for you to perceive, conceive and integrate those aspects of your experience that contradict the story; true because in acting out your story, which now includes the mark and its meaning, the social truth that created it is confirmed; true because if your mark is high you are consistently rewarded, so that your voice becomes a voice of authority in the power-knowledge discourses that reproduce the structure that helped to produce you; true because if your mark is low your voice becomes muted and confirms your lower position in the social hierarchy; true finally because that success or failure confirms that mark that implicitly predicted the now self-evident consequences. And so the circle is complete.”
In other words students “internalize” what those “marks” (grades/test scores) mean, and since the vast majority of the students have not developed the mental skills to counteract what the “authorities” say, they accept as “natural and normal” that “story/description” of them. Although paradoxical in a sense, the “I’m an “A” student” is almost as harmful as “I’m an ‘F’ student” in hindering students becoming independent, critical and free thinkers. And having independent, critical and free thinkers is a threat to the current socio-economic structure of society.
LikeLike
Sigh. He’s getting closer to saying the right things about testing (even though Harvard has been in the forefront of “reform” from the beginning), but he’s still blathering about needing “accountability” to measure student “achievement” and “performance” and to “close” the “achievement gap”, blah, blah, blah. Still all the same buzzwords proving, once again, the “best and brightest” just don’t get it.
LikeLike
I have a simple plan that provides significant financial incentive to attract and retain highly qualified teachers “where they are needed the most”
The plan is cost-free for high needs districts.
The TFT Plan:
After 3 years of service in a high needs school, a teacher then works “tax free”
Zero federal and zero state income taxes. The “Tax Free Teaching” benefit would follow the teacher as long as they remain in a HND. Current teachers would reap the same TFT benefit.
Uncle Sam could easily absorb the difference and it places no financial burden on the district. It has no increase on the pension burden either. I would imagine that about one million teachers would eventually qualify. Annual cost to the feds would be about 10 to 15 billion. This could be easily skimmed from the $600 billion military budget or by taxing the 10 million millionaires an extra one thousand dollars each.
LikeLike
Okay and I missing something here with respect to Quin Mulhollan’s commentary? Does it take a Harvard degree to state the obvious to just about anyone in public education these days?
LikeLike
Because lies about public education have profit potential and plutocratic sponsors, it’s rare for the contrary “obvious” message to find a spot in media space.
I appreciate Mulholland’s writing and the Review for publishing his work.
LikeLike