I posted a blog called “A Reader Wants to Know,” in which a teacher asked how he could be evaluated on test scores when the students were in the middle of the second semester. How were the scores affected by the teacher of the previous years? Other teachers have asked how they can be evaluated by scores when so many other factors affect test scores.
A reader commented on this blog with the most pertinent question of all. Why do we (and state legislatures and the U.S. Department of Education and the media) treat these tests and the scores they produce as accurate measures of what students know and can do? The reader, who clearly is a teacher, reminds us that the tests can’t do what everyone assumes they can do. They are subject to statistical error, measurement error, and random error. They are a yardstick that ranges from 30″ to 42″, sometimes more, sometimes less. Yet we treat them as infallible scientific instruments. They are not. He or she wrote:
In a way, I hate these questions. It’s in the same category as “Kid having a bad day on the day of the high stakes test”. They are all searching for explanations of why scores vary so widely, when the real answer is that all of these effects are dwarfed by the innate inaccuracy of the test. Even though these tests provide nice distinct numbers, there is a large random component to them.
To repeat: These tests are inherently inaccurate at the individual level.
Addressing anything but that just provides fodder for distraction for those who want to use them to loot our educational system.
This comment reminds me of something that has long occurred to me. The entire edifice of school-bashing and teacher-bashing relies on these shoddy tests. In the name of “accountability,” schools are being closed, people are being fired. And nothing is done to improve education. The students are shuffled from old school to new school, and in time they will be shuffled yet against from old new school to new new school.
What’s the game here and what does it have to do with improving education?
Diane
Diane Ravitch's blog 
we put so much time into differentiated instruction and then we think that miraculously a paper and pencil scantron test will tell us what the student knows….Really????
The “game” is what I refer to as “prescribed failure”. The tests are not meant to prove that we are doing what we, as educators, are suppose to be doing. But, instead they are being used as an instrument to validate the continued school bashing by politicians. Think about it. What gets parents more involved than the idea that their child is being mistreated, short-changed, or held back? If you don’t believe it, watch a kid’s sporting event. Those in power have figured out that this is a hot button for voters and have capitalized on it. Even though all statistics (using these inaccurate tests) prove that we are not lagging as far as the “fear mongers” would like everyone to believe (shout a lie loud enough and long enough and everyone will believe it), they continue to shout how our schools are not performing. I say that they fact that they keep raising the stakes ( ie rewriting the tests) in order to make it more difficult to pass, proves the lack of validity in this testing system. When will they stop “trying” to make us fail? The answer is when they have made sure that our kids “can’t ” pass this “dog and pony” show of a test so that they can then support their desire to privatize education and turn it into another money making corporation. Many parents have already proven that they will go into great dept to provide their children with a good education in the higher education arena. Now they want to put us all in more dept. This is a huge untapped, for the most part, source of income for banks and corporate figures who would bleed the middle class dry. And, we seem to be sitting by just watching it happen.
Standardized testing has become the root of all evil. Invalid test scores are used to shame and flunk students, fire teachers and close schools. Nationwide, billions of dollars are pumped into the standardized testing industry every year while schools simultaneously lay off teachers and cut enrichment programs and P.E. classes. And that is why we opt our children out of these tests. It is our right, as Americans, to participate in civil disobedience.
Question: Do the countries to which we are compared (ie Finland) use the same testing tools?
No!
Finland does not have standardized tests. Students take teachet-made tests.
The use of high stakes tests is a weapon held over the heads of teachers and schools, as are the value- added metrics used to evaluate teachers. Their faux scientific veneer is intended to make them impregnable to criticism, when in reality they are political devices used to justify the private takeover of the public schools, while enriching an ever- growing entourage of publishers, consultants, networks and profiteers.
The Common Core Standards are another vehicle to bring even more testing into the schools, which will become the pretext for yet more destabilization and privatization.
Yes. This is why Coleman is working for Michele Rhee!
Reblogged this on teachesol and commented:
I had my own frustrating experience this year with being judged based on high stakes test scores of my first and second year EL students. It’s sad because I thought they did so great and had a great year, but the numbers weren’t acceptable I guess.
The sad thing is that value-added measures are junk science, and yet they are being used to label, rank, and harm people’s lives.
I’m afraid I agree with Michael Fiorillo. The only reason for this nonsense is to undermine public education by failing students and blaming teachers. I see no evidence the “reformers” genuinely care even a little bit about this stuff. If they did, Bloomberg, Klein, Emmanuel, and Obama would send their kids to the test-prep factories they’re creating rather than private schools with small classes–which in fact are becoming even further out of reach for the ever-vanishing middle class.
What a tangled web they weave… Union-busting, teacher firings, and corporate take-overs of public education seem unlikely on the Democrat’s watch, but they do appear to be the major aims of the current “education reform” movement, regardless of party affiliation. And it all hinges on high-stakes testing, using measures that are not valid and reliable. I don’t think it ends there. I think there could be whole lot more to this –related to students, colleges and the work force that politicians and their coporate sponsors might be aspiring to create.
The insidious thing about it all –the Commom Core, narrowed curriculum, annual testing regime, and drill for skill teaching inheritantly required– is that this kind of schooling is most likely to produce automatrons, not creative people with higher order thinking skills. And yet, the current administration claims that the aim is for everyone to go to college. It’s hard to fathom this. I think that most students in this era who have been bombarded with 12 years of lockstep, assembly line schooling would be less likely to desire to continue on to college and/or to have the skills necessary to be successful there. Even if the majority of students do go to college, at most, many would be more likely to earn certificates, not degrees, at current colleges. Or are they trying to change what college is all about, particularly at the Community College level, since virtually all are public institutions –so they have the power to do that? If so, I wonder how that would impact articulation to four year schools, but with the federal government controling the purse strings (Title IV –financial aid), who knows what many colleges would do to comply with their demands? (This would be where the standards of regional accreditation and specialized professional associations must not waiver!)
So what might be the end game, especially for students? Could it be that politicians and corporations want to create a work force with large populations of people who they can claim are college educated, but who have virtually no critical thinking skills? Do they want workers who will be complacent, eager to take menial jobs for slave wages and be on call 24/7, so businesses don’t have to outsource anymore to places like China? Then politicians and corporations could say they brought jobs back to America, the worker bees here will be glad to have any jobs at all, and profits will rise for the 1%.
It sounds so 1984 –and very hard to stomach that anything like this could be happening in America, let alone under our first black president. I’m not sure how this might play out in detail. I’d love to hear what others think might be alternative plans and outcomes, because the thought of this is just so disheartening!
On a related note, ever since Pearson bought out so many other academic publishers, they seem to have had a strangle hold on education and I think you have to wonder about their role in all of this. (A cursory examination of their political contributions over the years suggests they have gone from being a major supporter of Republicans to supporting more Democrats after Obama took office.)
“It sounds so 1984 –and very hard to stomach that anything like this could be happening in America, let alone under our first black president.” Perhaps this is precisely why we have gotten to this point – people were so easily misled and distracted by Obama. It turns out, he was not transformational at all. The only thing different about him is his skin color. I’d like to think that if John McCain and Sarah Palin were in office trashing teachers and their unions, Democrats would fight back.
John Ewing (Math for America) calls the use of test scores, the way you describe it, as “mathematical intimidation”. More people need to know what these tests can and can’t do so that those in positions to challenge their misuse can continually point out better evidence (performance exhibitions, portfolios, Learning Records, and so forth).
Keep in mind that the tests usually samPle from the construct/content domain.
So, there’s some sampling error in there too.
There is especially sampling error if the tests are used to measure teacher
Or school quality.
[…] Diane Ravitch • Why Do We Treat the Tests as Scientific Instruments? Share this:TwitterFacebookLike this:LikeBe the first to like this post. This entry was posted in […]
All test score input is contrived. Pearson ,et al, monitor scores to be sure they “fit” the expected range.
What’s the game here?
Common Core, privatization, teacher bashing, union busting are not end results, merely means to an end. The end results:
* Justification for a national data base on “student achievement”. (Very scary.)
* Deskill public education, saving taxpayers about 1 billion dollars a year. (Higher education for masses not necessary to compete in global, 21st century market.)
* End around to Brown vs. Board of Education 1954. (Supremacy cannot be attained through law, so instead by commodification, including public education.)
Remember, the purpose of public education was NEVER to create autonomous, high level, life-long learners. Public education was created so that corporate America would have the necessary work force to drive its engine. Education reform is not “reforming” education per se, merely modifying and limiting public education to continue to meet its original purpose.
[…] Why Do We Treat the Tests As Scientific Instruments? « Diane Ravitchs blog. Share this:ShareLike this:LikeBe the first to like this […]
Remember that if there’s a question that all the kids get right, no one says, “Hooray, our kids are learning and our teachers are teaching!”
Instead they say, “Huh, guess that one was too easy,” and remove it from next year’s test.
Neither these tests–nor the scoring of these tests–are standard, valid or reliable.
Once again, I must strongly recommend reading “Making the Grades” (I apologize for quotation marks rather than italics or underling for italics, which is proper for a book title–I can’t get mine to work w/in the context of a blog comment, but I do know the difference!) by Todd Farley–2009. Also, read any Pearson Help Wanted ad for further validation of aforementioned!
Standardized testing is the trojan horse of the information war. The most important things I have ever learned will never appear on a standardized test. The most important things I have ever taught will never appear on a standardized test.
Diane, I’m so very happy to hear that you will be appearing tomorrow night (June 5) on the PBS NewsHour.
I’m curious about the statement made by Melinda Gates, tonight when she appeared on that same show this evening and I was wondering what you thought about it.
Here is her statement, from the transcript:
MELINDA GATES: “Well, we know from good research that the fundamental thing that makes a difference in the classroom is an effective teacher. An effective teacher in front of a student, that student will make three times the gains in a school year that another student will make.”
Do you have any idea to what research is Ms. Gates referring? I’d be curious to know. I don’t mean to be offensive nor rancorous in asking this question. But I would like to know more about this study, how its data was collected, its subjects, and so forth. And again, Diane, I’d like to hear your thoughts on all of this.
Thanks.
That quote is tossed around a lot by folks in the corporate education “reform” movement and can be found in the first paragraph here, Breakthrough Collaborative Sept 2011 Research Brief. It’s attributed to Hanushek, E. (2010) “The Difference is Teacher Quality.” In Waiting for Superman: How We Can Save America’s Failing Public Schools. As cited in Center for Analysis of Longitudinal Education Research (2010) Fact Sheet: The Evidence Behind Waiting for Superman.
In Diane’s book, The Death and Life of the Great American School System, she addressed the research the conclusions drawn by Hanushek and others, which Valerie Strauss quoted extensively in this piece in the Washington Post.
Sorry, try this link to the Brief mentioned above: http://www.breakthroughcollaborative.org/sites/default/files/Sept%202011%20Research%20Brief-All%20Eyes%20on%20Teachers_1.pdf
That’s a myth. No research shows that. Her people should brief her better.