When the purveyors of evaluation systems are hawking their latest program, they confidently assert that the test scores are only one of multiple measures.
Don’t worry, they say, the test scores are only 20%, or 30%, or 40%, or 50%.
We will put them into context with lots of soft measures derived from classroom observations or other non-data sources.
But it is not true, even if they mean it when they say it.
This principal writes:
The allure of data is simply too strong to resist.
I began teaching in a state where a high-stakes state testing system was already in place. Naturally, the pressure was intense on teachers, schools, and districts, but it was the only reality I knew.
Later, my wife and I moved to another state where high-stakes tests were just a seed of an idea, almost beginning to germinate. Of course, there was talk of it being just “one of many indicators.” Both my wife and I thought, “Yeah, right. We know what’s coming.” And ten plus years later, this fleur du mal has fully bloomed.
As a principal, I try to use data sensibly, as just “one of many indicators,” but it’s a losing battle. It’s difficult to complicate people’s thinking, to point out the complexities of educating young people, in an environment hooked on numbers. They’re so tangible, so easy to communicate. We can wave them about as proof of success or failure. As complicated as the algorithm can be, the numbers dumb us down.
I need our numbers to show improvement or else my leadership is questioned (or I’ll possibly get turnarounded). I don’t need to point out the numbers to our teachers; they scour them, searching for vindication. Our community judges us by them. As for the politicians and policymakers, we know how they use them.
I can’t go so far as to say the numbers have no place in education (Why can’t I go so far? Maybe I need to challenge my thinking on this.), but it appears that consuming them in moderation demands a level of intellectual rigor and self-discipline we don’t possess.
Diane Ravitch's blog 
I think the problem here is that the “other” measures aren’t really viewed as data. Of course they are, but our fascination with the statistical measures trumps all else. But I think a start would be talking about it as statistical (or testing data) and qualitative or observational data. If we viewed it all as data we’d be off to better start. Of course I must be dreaming thinking that will happen in the current environment.
Well, then,I’m probably just dreaming, too, because, this week, when Arne Duncan asked on Twitter what it would take to transform the field, I replied with tweets about project-based learning and portfolio assessment. Even included a link to the Work Sampling System –thought he might consider that since it’s published by Pearson (and it includes portfolios). Also asked him if he’d learned the value of qualitative research when he studied Sociology. I don’t know how he could not have learned that at Harvard.
I had already broached the topic earler this week, too. Sent him stuff about the importance of relationship-based learning and asked how that was going to be assessed with standardized tests.
I doubt he reads my tweets, but, hey, he will often ask a question like that, so I answer –and hope that somewhere inside that brain of his he is open to learning something from real educators. Yeah, probably just dreaming…
I wonder if he writes the tweets himself or even sees what he is supposed to have written.
What we need is science teachers to stand up across this country and help people understand what numbers are and what they do. Educational research based on statistical data can only be validated based on their reliability, which relies on correlation derivations. This sort of math and science is NOT the sort of math and science used in traditional (natural) science laboratory settings, or if they are used, they are not used within the same context and are used within methods that are by default valid. This is not so with the social sciences. The reformers are waving this data around concluding certain things that they have no right (and even including their proposition that the data is valid), but because this “soft” sort of science has become so ingrained as “truth” in this country, people tend to believe it. Therefore, if students of a teacher perform poorly on a standardized test, people flurry to point fingers based on the data without thinking about the actual science used. It’s like taking what a meteorologist says as gospel truth. RIchard Feynman said it best that in carrying out science, we should be “doing whatever it takes to avoid being fooled by reality”. These economists, econometricians, psychometricians, psychologists, and sociologists are “fooled by reality” much easier than real scientists. And they have taken over education, while they collect their millions. While their work may be admirable, as is the work of a meteorologist, people need to understand the limitation of their work, especially in the social context in which placing values and variables on the worth of people can be catastrophic.
Politicians pick and choose when data/accountability will best suit their needs. At yesterday’s hearing at the Capitol (Austin, Texas), voucher salesmen come up with contradictory strategies:
“The out-of-state advocates of private school vouchers cautioned against extending the state’s testing and accountability system to the private schools because it has failed to improve public schools.”
http://www.statesman.com/news/texas-politics/charter-school-supporters-debate-adding-charters-accountability-2441847.html
Evidently data is the weapon of choice for punishing schools but doesn’t lead to improvement. And the only way to avoid punishment is to choose homeschool, private school or charter school.
Politicians pick and choose when data/accountability will best suit their needs.
Got that right. The media does, too!
Snake Oil. The Data Machine of corporate takeover agendas purport a “softening” of proposed evaluation frameworks. I heard another politician tell the viewers of a major network TV broadcast that they would “soften” their confirmed vision of control on women’s reproductive rights. Forceable anything is wrong.
Aside from making way too much out of numbers generated from a single test on a single day, the use of that data is becoming increasingly questionable. In Massachusetts, the DESE has just released “guidance” on the use of Student Growth Percentiles (SGPs) in teacher evaluation. Growth below the 40th percentile is considered “low growth.” The DESE’s guidance for district-developed assessments is to calculate the difference between pre- and post-test results (growth), list the results in order, from highest to lowest, and divide the list into three equal bands. The lowest third is considered “low growth.” Thus, in both cases, no matter how well students are learning, 40% (on state assessments) or one-third (on district assessments) will always be considered as having “low growth” and this will be factored into teacher evaluations. ????!!!!! Does this make any sense at all?
There is quantitative data and qualitative data, and both should be identiifed, used, and respected as valid.
However, edreform is looking for the easy answer, the easy way to rate, rank, and sort students, educators, and workers. It is much easier and appears so much more clear-cut and soild to say that Johnny scored 625 on the state test, not meeting his target of 682, and only 48% of Mrs. McGillicuddy’s students scored in the acceptable range so she is judged less than effective. Qualifying the teacher through observation is time-consuming and seems problematic.
Recent conversations have more than hinted that some suspect that ineffective teachers during an announced observation may be able to somehow mesmerize an observer with a well-prepared and well-executed lesson, thereby covering up their flaws and weaknesses. This is hard for me to believe. Such a move would take incredible skill and effort.
“I can’t go so far as to say the numbers have no place in education (Why can’t I go so far? Maybe I need to challenge my thinking on this.), but it appears that consuming them in moderation demands a level of intellectual rigor and self-discipline we don’t possess.”
Consuming educational “data” in “moderation” is like consuming meth or crack cocaine in moderation-it is hardly ever done.