Apparently in response to Daniel Denvir’s article accusing the state of Pennsylvania of dragging its feet on an investigation of cheating on state tests, the state released some information.
Just remember whenever you hear a governor or other politician boasting about test scores that they have no idea (and neither will you) whether the gains were produced by mindless test prep (where students learn how to guess the right answer), by teaching to the test (where teachers abandon their professional ethics), or by cheating (by teachers or principals or students or superintendents or the state education department).
In a high-stakes environment, the pressure to raise scores renders them meaningless. See “Campbell’s Law.”
Diane Ravitch's blog 
It just hit me: high stakes testing is the ‘performance-enhancing drugs” of education.
If so, Michelle Rhee is Lance Armstrong
In keeping with today’s big sports cheating story, more like Ryan Braun or Alex Rodriguez. But Lance will always do in a pinch.
Definitely Lance Armstrong. Lance used a lot of threats, and other sorts of bullying to intimidate people from speaking up and to ruin people who did, just like Rhee.
Nah, PEDs are meant to enhance body strength which subsequently enhances performance-usually athletic, whereas high stakes testing destroys performance (the teaching and learning process.) Ask most post secondary students and they’ll tell you the PEDs for academic performance (note I didn’t say for the teaching and learning process) are those amphetamine mixed salts drugs like Adderall
Professor John Seddon gives a great critique of Sir Michael Barber’s “Deliverology”. If you have time, watch him on the link below. Common Core testing tied to teacher evaluations is an example of this top down, command and control, set targets and punish as the targets are not met system. Seddon goes around the world helping companies evaluate success from the customer’s point of view and creating efficiency based on that… which ends up improving morale and profits. Setting arbitrary targets from the top always demoralizes staff, innovates ways to “game the system”, costs more and doesn’t work. He suggests reform always starts by asking the customer, in the case of education, the student, what they need and want. How different would our schools be if we took this approach?
http://vimeo.com/11896519
Yes! I, too, stumbled across the engaging and mind-broadening Professor Seddon just yesterday on YouTube and Vimeo as I was reading something dreary about Barber and “Deliverology.” As Seddon discussed the importance of understanding systems and what value looks like to those on the receiving end, I thought of the work of Marion Brady with whom, I admit, I was not familiar until recently. A spark! A connection! Will study further….
“In a high-stakes environment, the pressure to raise scores renders them meaningless. See ‘Campbell’s Law'”
Campbell’s Law is not the guiding precedent in “rendering them meaningless. And, likewise, the “pressure to raise scores” isn’t the basic/root reason why the scores are “meaningless”.
The basic/radical/root reason that the scores are meaningless lies in the fact that there are so many logical errors in the processes that lead to those scores, that is the making of educational standards and standardized tests (and even the “grading” of students) and the giving of and disseminating the results of the tests that the processes are completely logically invalid epistemologically* and ontologically** which render the results “vain and illusory” and/or without meaning. Wilson has proven these invalidities that “render them meaningless” in “Educational Standards and the Problem of Error” found at: http://epaa.asu.edu/ojs/article/view/577/700 .
Every single person involved in public education should read and understand what Wilson has to say and realize the vast harm caused to so many students, and by extension the teachers and schools that fall under these testing regimes, by such sorting and separating and labeling educational malpractices.
Brief outline of Wilson’s “Educational Standards and the Problem of Error” and some comments of mine. (updated 6/24/13 per Wilson email)
1. A quality cannot be quantified. Quantity is a sub-category of quality. It is illogical to judge/assess a whole category by only a part (sub-category) of the whole. The assessment is, by definition, lacking in the sense that “assessments are always of multidimensional qualities. To quantify them as one dimensional quantities (numbers or grades) is to perpetuate a fundamental logical error” (per Wilson). The teaching and learning process falls in the logical realm of aesthetics/qualities of human interactions. In attempting to quantify educational standards and standardized testing we are lacking much information about said interactions.
2. A major epistemological mistake is that we attach, with great importance, the “score” of the student, not only onto the student but also, by extension, the teacher, school and district. Any description of a testing event is only a description of an interaction, that of the student and the testing device at a given time and place. The only correct logical thing that we can attempt to do is to describe that interaction (how accurately or not is a whole other story). That description cannot, by logical thought, be “assigned/attached” to the student as it cannot be a description of the student but the interaction. And this error is probably one of the most egregious “errors” that occur with standardized testing (and even the “grading” of students by a teacher).
3. Wilson identifies four “frames of reference” each with distinct assumptions (epistemological basis) about the assessment process from which the “assessor” views the interactions of the teaching and learning process: the Judge (think college professor who “knows” the students capabilities and grades them accordingly), the General Frame-think standardized testing that claims to have a “scientific” basis, the Specific Frame-think of learning by objective like computer based learning, getting a correct answer before moving on to the next screen, and the Responsive Frame-think of an apprenticeship in a trade or a medical residency program where the learner interacts with the “teacher” with constant feedback. Each category has its own sources of error and more error in the process is caused when the assessor confuses and conflates the categories.
4. Wilson elucidates the notion of “error”: “Error is predicated on a notion of perfection; to allocate error is to imply what is without error; to know error it is necessary to determine what is true. And what is true is determined by what we define as true, theoretically by the assumptions of our epistemology, practically by the events and non-events, the discourses and silences, the world of surfaces and their interactions and interpretations; in short, the practices that permeate the field. . . Error is the uncertainty dimension of the statement; error is the band within which chaos reigns, in which anything can happen. Error comprises all of those eventful circumstances which make the assessment statement less than perfectly precise, the measure less than perfectly accurate, the rank order less than perfectly stable, the standard and its measurement less than absolute, and the communication of its truth less than impeccable.”
In other word all the errors involved in the process render any conclusions invalid.
5. The test makers/psychometricians, through all sorts of mathematical machinations attempt to “prove” that these tests (based on standards) are valid-errorless or supposedly at least with minimal error [they aren’t]. Wilson turns the concept of validity on its head and focuses on just how invalid the machinations and the test and results are. He is an advocate for the test taker not the test maker. In doing so he identifies thirteen sources of “error”, any one of which renders the test making/giving/disseminating of results invalid. As a basic logical premise is that once something is shown to be invalid it is just that, invalid, and no amount of “fudging” by the psychometricians/test makers can alleviate that invalidity.
6. Having shown the invalidity, and therefore the unreliability, of the whole process Wilson concludes, rightly so, that any result/information gleaned from the process is “vain and illusory”. In other words start with an invalidity, end with an invalidity (except by sheer chance every once in a while, like a blind and anosmic squirrel who finds the occasional acorn, a result may be “true”) or to put in more mundane terms shit in-crap out.
7. And so what does this all mean? I’ll let Wilson have the second to last word: “So what does a test measure in our world? It measures what the person with the power to pay for the test says it measures. And the person who sets the test will name the test what the person who pays for the test wants the test to be named.”
In other words it measures “’something’ and we can specify some of the ‘errors’ in that ‘something’ but still don’t know [precisely] what the ‘something’ is.” The whole process harms many students as the social rewards for some are not available to others who “don’t make the grade (sic)” Should American public education have the function of sorting and separating students so that some may receive greater benefits than others, especially considering that the sorting and separating devices, educational standards and standardized testing, are so flawed not only in concept but in execution?
My answer is NO!!!!!
One final note with Wilson channeling Foucault and his concept of subjectivization:
“So the mark [grade/test score] becomes part of the story about yourself and with sufficient repetitions becomes true: true because those who know, those in authority, say it is true; true because the society in which you live legitimates this authority; true because your cultural habitus makes it difficult for you to perceive, conceive and integrate those aspects of your experience that contradict the story; true because in acting out your story, which now includes the mark and its meaning, the social truth that created it is confirmed; true because if your mark is high you are consistently rewarded, so that your voice becomes a voice of authority in the power-knowledge discourses that reproduce the structure that helped to produce you; true because if your mark is low your voice becomes muted and confirms your lower position in the social hierarchy; true finally because that success or failure confirms that mark that implicitly predicted the now self evident consequences. And so the circle is complete.”
In other words students “internalize” what those “marks” (grades/test scores) mean, and since the vast majority of the students have not developed the mental skills to counteract what the “authorities” say, they accept as “natural and normal” that “story/description” of them. Although paradoxical in a sense, the “I’m an “A” student” is almost as harmful as “I’m an ‘F’ student” in hindering students becoming independent, critical and free thinkers. And having independent, critical and free thinkers is a threat to the current socio-economic structure of society.
*Epistemology (from Greek ἐπιστήμη – epistēmē, meaning “knowledge, understanding”, and λόγος – logos, meaning “study of”) is the branch of philosophy concerned with the nature and scope of knowledge[1][2] and is also referred to as “theory of knowledge”. It questions what knowledge is and how it can be acquired, and the extent to which any given subject or entity can be known. (from Wiki)
** Ontology is the philosophical study of the nature of being, becoming, existence, or reality, as well as the basic categories of being and their relations. Traditionally listed as a part of the major branch of philosophy known as metaphysics, ontology deals with questions concerning what entities exist or can be said to exist, and how such entities can be grouped, related within a hierarchy, and subdivided according to similarities and differences. (from Wiki)
As usual, spot-on Duane. In short, these “standardized” tests are neither valid nor reliable–therefore, not standardized. Add in errors in computerized test-taking (Indiana glitches, for example) and juked scoring (or just plain incompetent scoring OR invalid computer scoring)…WHAT is it, then, that the powers-that-be are placing everything on?!
Everyone, sign the Resolution to End High Stakes Testing but–most important–OPT OUT NOW. No one to take the tests=No data, no VAM, no school closings based on “poor” test scores/failure to make AYP–it’s over and done.
Will the SSA end high stakes testing or just remove the Federal Government’s oversight of high stakes testing, but still require states to administer these tests?
CM,
What is SSA?
Thanks,
Duane
http://edworkforce.house.gov/studentsuccessact/
Student Success Act to replace NCLB. I am trying to read through it, but I find many bills confusing and take a while to fully understand all the ins and outs.
It is a done deal now. Below is a summary of some good things. Unfortunately the bill also give military recruiters the same access to high schools they now have to colleges. It will need a companion bill in the senate, maybe we can rally to get that taken out there.
As passed by the House, July 19th, HR 5, the Student Success Act will:
Eliminate AYP and replace it with state-determined accountability systems, thereby returning authority for measuring student performance to states and school districts.
Eliminate federally mandated actions and interventions currently required of poor performing schools, giving states and districts maximum flexibility to develop appropriate school improvement strategies and rewards for their schools.
Allow Title I dollars to follow disadvantaged kids, at the state option. This landmark ‘portability’ option promotes parental choice and allows all Title I schools to receive funds to promote the academic achievement of students in need.
Repeal federal “Highly Qualified Teacher” requirements and grant states the flexibility to develop their own teacher evaluation systems to better gauge an educator’s effectiveness, if they so choose.
Maintain the requirement that states and school districts issue and distribute annual report cards, including disaggregated data on student achievement and high school graduation rates, while also streamlining data reporting to ensure meaningful information is easily available to parents and communities.
Support opportunities for parents to enroll their children in local magnet schools and charter schools, and enhance statewide parental engagement.
Eliminate more than 70 existing elementary and secondary education programs to promote a more appropriate federal role in education.
Consolidate a myriad of existing K-12 education programs into a new Local Academic Flexible Grant, which provides funding to states and school districts to support local priorities that improve student achievement.
Protect state and local autonomy over decisions in the classroom by limiting the authority of the secretary of education, including by eliminating the secretary’s ability to inappropriately influence state decisions to adopt the Common Core or other common standards or assessments
Generally an improvement, but I agree, Dawn, about keeping military recruiters off of high school campuses.
That’s a good question, concerned. I think there’s something specific about that in the ESEA, and I think that’s what’s being proposed. Is this the case, Diane?
Retiredbutmissthekids, I would answer your question but don’t know what it is
Diane, it’s originally concerned mom’s question of July 23rd at 7:18 PM. But I think Duane answered it, in part. Further question, though–is the Student Success Act part of the ESEA? All of this legislation makes me want to scream–it’s on the order of “If you can’t dazzle ’em with your brilliance, baffle ’em with your b.s.”
Retiredbutmissthekids,
The basic federal education law was passed and signed into law by President Lyndon Johnson in 1965. It was called the Elementary and Secondary Education Act. That was in the days when Congress called laws what they were, rather than giving them flowery and silly names.
No Child Left Behind was Gorge W. Bush’s revision of ESEA. All federal laws are supposed to be re authorized every seven years. NCLB was passed in 2001 and sined into law in January 2002. It should have been reauthorized years ago but Congress is frozen. It knows NCLB was a failure but can’t figure out to change it without someone yelling that they are watering down “accountability.” Of course, the accountability is the worst, most punitive and ineffective part of NCLB.
The Student Success Act was written by House Republicans, who want to restore local control.
The Democrats control the Senate nd they will write their own bill to reflect what Obama wants.
Which ironically looks very much like NCLB.
So will high stakes testing remain? Is an opt-out option be something we could try to get on a ballot?
We need to call our House Members and show our support for the Student Success Act and thank those who supported it. We need to call our Senators and request that they support the Student Success Act much more than we do a re-named NCLB bill.