Stephen Dyer in Ohio can’t believe that 2/3 of the children in New York failed the state tests. He says if he or his wife wrote an exam that 69% of students failed, the shame would be on them or the tests, not the students.
The Néw York scores lack face validity.
Dyer says: “Does anyone really, I mean really, believe that more than 2 out of every 3 children in New York State are failing? Or that only 5% of some subsets pass? Or that the schools in New York State (which consistently rank pretty well in EdWeek’s rankings) are really that bad?”
And he adds: “High standards don’t mean that more than 2 out of 3 kids have to fail. High standards and normal test scores are perfectly compatible.”
Maybe, he suggests the problem is the tests, not the kids.
Diane Ravitch's blog 
Diane
Your comments on PISA rankings problems would be helpful especially the implications for teachers via the OECD TALIS project http://www.tes.co.uk/article.aspx?storycode=6347169
http://susanohanian.org/data.php?id=510
The fault, Dear Brutus, is not in our stars, but in our reformy tests.
Shame on the students, no they are wanting to put the shame on the teachers.
This is what happens when the People let power fall into the hands of fools.
It is the very reason that education — and respecting education — is essential to the life of democracy.
Without it democracy dies.
Anyone not living in Gate’,s Jeb’s, etc. dream world knows that the problem lies with the tests, not the kids. We’ve got to keep fighting this insanity and get EVERYONE involved. Part of the fight must be to get education friendly candidates into EVERY office.
True education is a long game. The problem with the Common Core tests may not be the tests or the standards: we have no basis for judgement because the standards and benchmarks for performing well on those standards are new. It takes time to teach to new standards. Teachers need time to plan new lessons devoted to new skills, to practice those lessons, and then to revise those lessons based on student performance. Education is cyclical process; it is a time-consuming process; and, much to the corporate reformers’ displeasure, it is a messy process (and that’s not a bad thing).
If the people behind these tests genuinely understood what it means to educate a person, if they believed in the integrity of both the standards and the tests, they would insist that only those children who are taught in accordance to those standards from the moment they set foot in school would be tested. For many states, this would mean that the current crop of first graders and those who follow them would be the only students tested because they are going to be the only students who receive an education completely dictated by those standards.
Of course, students are failing these tests: they haven’ t been adequately prepared for them. And the only people to blame for this are the politicians, corporate-minded-philanthropists, and business leaders, most of whom have no long-term or meaningful teaching experience in the public school system, to make decisions about how to educate our children.
Thank you Jessica for this thoughtful and accurate assessment of the situation.
We were all forewarned that students nationwide were going to fail this testing since teachers have not yet been thoroughly trained and had time to teach to the Common Core standards….so why is everyone surprised at the result?
That’s one of the reasons that I have a huge problem with the CC implementation. They’ve dropped kids into the middle of these new standards with no foundation and no remedial help and then expect them to somehow figure it out on their own. It’s ridiculous.
Ellen,
“Thank you Jessica for this thoughtful and accurate assessment of the situation.”
Yes it is an accurate assessment (interesting that said assessment has nothing to do with being “standardized” and is as idiosyncratic as all assessments”
But it begs the question of whether these standardized standards and tests are logical, good, necessary, desirable, achievable etc. . . . See my responses below/above to Jessica to see why.
Disagree,
This leaves us with some idea that the curriculum is useful and valid. This is the Kool Aid that the unions are pushing. “We just need more time”, “We don’t have the materials” New York State has (had) a wonderful curriculum which opened the door for concepts and not the rote learning of facts and handouts. I am still searching for the Common Core section on the rise of labor unions. Maybe the teachers union can find it when they get out of bed with the publishers. Their members are expert in hand outs by the publishers that begin nowhere and end nowhere.
“The problem with the Common Core tests may not be the tests or the standards: we have no basis for judgement because the standards and benchmarks for performing well on those standards are new. It takes time to teach to new standards.”
No, no, no, no, no!!! (said in a teacher’s exasperated voice)
Yes, the problems are the educational standards and standardized tests. You have hit on the problem of “doing the wrong thing righter”.
The proliferation of educational assessments, evaluations and canned programs belongs in the category of what systems theorist Russ Ackoff describes as “doing the wrong thing righter. The righter we do the wrong thing,” he explains, “the wronger we become. When we make a mistake doing the wrong thing and correct it, we become wronger. When we make a mistake doing the right thing and correct it, we become righter. Therefore, it is better to do the right thing wrong than the wrong thing right.”
Our current neglect of instructional issues are the result of assessment policies that waste resources to do the wrong things, e.g., canned curriculum and standardized testing, right. Instructional central planning and student control doesn’t – can’t – work. But, that never stops people trying.
The result is that each effort to control the uncontrollable does further damage, provoking more efforts to get things in order. So the function of management/administration becomes control rather than creation of resources. When Peter Drucker lamented that so much of management consists in making it difficult for people to work, he meant it literally. Inherent in obsessive command and control is the assumption that human beings can’t be trusted on their own to do what’s needed. Hierarchy and tight supervision are required to tell them what to do. So, fear-driven, hierarchical organizations turn people into untrustworthy opportunists. Doing the right thing instructionally requires less centralized assessment, less emphasis on evaluation and less fussy interference, not more. The way to improve controls is to eliminate most and reduce all.
Former Green Beret Master Sergeant Donald Duncan (Viet Nam) did when he noted in Sir! No Sir! that:
“I was doing it right but I wasn’t doing right.”
And from one of America’s premier writers:
“The mass of men [and women] serves the state [education powers that be] thus, not as men mainly, but as machines, with their bodies. They are the standing army, and the militia, jailors, constables, posse comitatus, [administrators and teachers], etc. In most cases there is no free exercise whatever of the judgment or of the moral sense; but they put themselves on a level with wood and earth and stones; and wooden men can perhaps be manufactured that will serve the purpose as well. Such command no more respect than men of straw or a lump of dirt.”- Henry David Thoreau [1817-1862], American author and philosopher
And (doing my best Don Quixote de la Mancha imitation):
Jessica,
Read and learn why you are incorrect in your thinking:
“Educational Standards and the Problem of Error” found at: http://epaa.asu.edu/ojs/article/view/577/700
Brief outline of Wilson’s “Educational Standards and the Problem of Error” and some comments of mine. (updated 6/24/13 per Wilson email)
1. A quality cannot be quantified. Quantity is a sub-category of quality. It is illogical to judge/assess a whole category by only a part (sub-category) of the whole. The assessment is, by definition, lacking in the sense that “assessments are always of multidimensional qualities. To quantify them as one dimensional quantities (numbers or grades) is to perpetuate a fundamental logical error” (per Wilson). The teaching and learning process falls in the logical realm of aesthetics/qualities of human interactions. In attempting to quantify educational standards and standardized testing we are lacking much information about said interactions.
2. A major epistemological mistake is that we attach, with great importance, the “score” of the student, not only onto the student but also, by extension, the teacher, school and district. Any description of a testing event is only a description of an interaction, that of the student and the testing device at a given time and place. The only correct logical thing that we can attempt to do is to describe that interaction (how accurately or not is a whole other story). That description cannot, by logical thought, be “assigned/attached” to the student as it cannot be a description of the student but the interaction. And this error is probably one of the most egregious “errors” that occur with standardized testing (and even the “grading” of students by a teacher).
3. Wilson identifies four “frames of reference” each with distinct assumptions (epistemological basis) about the assessment process from which the “assessor” views the interactions of the teaching and learning process: the Judge (think college professor who “knows” the students capabilities and grades them accordingly), the General Frame-think standardized testing that claims to have a “scientific” basis, the Specific Frame-think of learning by objective like computer based learning, getting a correct answer before moving on to the next screen, and the Responsive Frame-think of an apprenticeship in a trade or a medical residency program where the learner interacts with the “teacher” with constant feedback. Each category has its own sources of error and more error in the process is caused when the assessor confuses and conflates the categories.
4. Wilson elucidates the notion of “error”: “Error is predicated on a notion of perfection; to allocate error is to imply what is without error; to know error it is necessary to determine what is true. And what is true is determined by what we define as true, theoretically by the assumptions of our epistemology, practically by the events and non-events, the discourses and silences, the world of surfaces and their interactions and interpretations; in short, the practices that permeate the field. . . Error is the uncertainty dimension of the statement; error is the band within which chaos reigns, in which anything can happen. Error comprises all of those eventful circumstances which make the assessment statement less than perfectly precise, the measure less than perfectly accurate, the rank order less than perfectly stable, the standard and its measurement less than absolute, and the communication of its truth less than impeccable.”
In other word all the errors involved in the process render any conclusions invalid.
5. The test makers/psychometricians, through all sorts of mathematical machinations attempt to “prove” that these tests (based on standards) are valid-errorless or supposedly at least with minimal error [they aren’t]. Wilson turns the concept of validity on its head and focuses on just how invalid the machinations and the test and results are. He is an advocate for the test taker not the test maker. In doing so he identifies thirteen sources of “error”, any one of which renders the test making/giving/disseminating of results invalid. As a basic logical premise is that once something is shown to be invalid it is just that, invalid, and no amount of “fudging” by the psychometricians/test makers can alleviate that invalidity.
6. Having shown the invalidity, and therefore the unreliability, of the whole process Wilson concludes, rightly so, that any result/information gleaned from the process is “vain and illusory”. In other words start with an invalidity, end with an invalidity (except by sheer chance every once in a while, like a blind and anosmic squirrel who finds the occasional acorn, a result may be “true”) or to put in more mundane terms shit in-crap out.
7. And so what does this all mean? I’ll let Wilson have the second to last word: “So what does a test measure in our world? It measures what the person with the power to pay for the test says it measures. And the person who sets the test will name the test what the person who pays for the test wants the test to be named.”
In other words it measures “’something’ and we can specify some of the ‘errors’ in that ‘something’ but still don’t know [precisely] what the ‘something’ is.” The whole process harms many students as the social rewards for some are not available to others who “don’t make the grade (sic)” Should American public education have the function of sorting and separating students so that some may receive greater benefits than others, especially considering that the sorting and separating devices, educational standards and standardized testing, are so flawed not only in concept but in execution?
My answer is NO!!!!!
One final note with Wilson channeling Foucault and his concept of subjectivization:
“So the mark [grade/test score] becomes part of the story about yourself and with sufficient repetitions becomes true: true because those who know, those in authority, say it is true; true because the society in which you live legitimates this authority; true because your cultural habitus makes it difficult for you to perceive, conceive and integrate those aspects of your experience that contradict the story; true because in acting out your story, which now includes the mark and its meaning, the social truth that created it is confirmed; true because if your mark is high you are consistently rewarded, so that your voice becomes a voice of authority in the power-knowledge discourses that reproduce the structure that helped to produce you; true because if your mark is low your voice becomes muted and confirms your lower position in the social hierarchy; true finally because that success or failure confirms that mark that implicitly predicted the now self evident consequences. And so the circle is complete.”
In other words students “internalize” what those “marks” (grades/test scores) mean, and since the vast majority of the students have not developed the mental skills to counteract what the “authorities” say, they accept as “natural and normal” that “story/description” of them. Although paradoxical in a sense, the “I’m an “A” student” is almost as harmful as “I’m an ‘F’ student” in hindering students becoming independent, critical and free thinkers. And having independent, critical and free thinkers is a threat to the current socio-economic structure of society.
Thank you for sharing Wilson’s work. http://epaa.asu.edu/ojs/article/view/577/700
Wonder if I’ll have time to read it all before school starts…
My NYS 8th graders said it best regarding the NYSE Common Core ELA Exam, ” It’s not that the test was even that hard. It was too long, and we didn’t get enough time to prove what we know.”
All the $$ going to Pearson is so sad. All the money being spent on the wrong “solutions” resulting in larger class sizes and teacher layoffs is disheartening. My position was excised even after being in my district for 7 years. The humor in this is my students state exam scores were the highest in the district, and I am the teacher who parents beg to have their child be put in my class.
“. . . I am the teacher who parents beg to have their child be put in my class.”
Do you have some hard data to verify that????
(ok, I turned off my sarcasmometer now)
John Taylor Gatto in his “Underground History of Education”, even Amazon won’t carry it, identifies the system as not tolerating popular teachers, a real death knell.
That book will turn your head around.
face validity… is not validity evidence. It is an old idea that testing has long seen as useless- it is a mirage.
Ryan, these tests lack any kind of validity. State officials knew the results before the kids took the test. Isn’t that amazing?
I’ve worked in many schools where far fewer than 69% were competent in multiple subjects based on multiple assessments. I’d be interested in what Mr. Dyer is basing his observations on. I read his blog post but saw no references to other data indicating that academic achievement is actually higher.
Please note I’m not saying that the test is too hard or easy, but we really should hold ourselves to a more rigorous method of evaluation than, “I can’t believe test scores aren’t higher than 69%, so the test must be faulty.” What if I find you someone who believes that test scores should really be much lower – would you post that person’s opinion as evidence that the test was overestimating achievement?
Personally, I’ve worked in many schools in which 69% would be much higher than actual achievement rates, as measured by multiple assessments across multiple subjects. I wouldn’t presume, though, to be an expert on NY public education achievement based on my own personal experiences.
The short: If the tests are bad, let’s stick to actual evidence that they are bad, rather than posting someone’s opinion that cites no evidence other than, “I can’t believe this, so it must not be true.”
Yes, 50 – 70% competent might be expected in many assessments of material taught. 69% failing (the results reported for NYS CC tests), on the other hand, exclusive of any other information, demonstrates that the assessment was based largely on material not taught.
Spanish&French Freelancer – thanks for pointing that out. I totally misread the initial post as 69% passing. I would agree that 31% passing definitely seems more suspect, especially across an entire state. Thanks for the correction.
Edededucation, you will love my new book. Lots for you to think about. You might even change your mind once you have seen the evidence.
Thanks Diane – I’ll definitely look for it. In terms of changing my mind, though, I imagine we already agree on most topics related to state testing.