Rocky Killion is an amazing superintendent in West Lafayette, Indiana. To begin with, he produced a wonderful documentary about the assault on public education, called “Rise Above the Mark.” You can go to the website to find out how to order a copy to show in your community (it is also for sale on amazon.com). He is very critical of the testing-gone-wild culture that has been foisted on public schools in Indiana and across the nation. He is very sensitive to the damage done to education, to children, and to teachers. His colleagues named him Indiana’s Superintendent of the Year for 2015.
Now he is furious because the computers that give the state test–the ISTEP–froze during a practice run. That was just too much.
“It’s inhumane what we are doing to the kids, what we are doing to the educational environment, we lost so much instructional time today, it’s ridiculous,” Killion told WTHR-TV in Indianapolis on Feb. 12, after computers froze during a dry run for ISTEP last week.
The Superintendent of the Year for 2015, as named by the Indiana Association of Public School Superintendents, followed it with this: “I would prefer all of my students’ parents withdraw and become home-schooled during ISTEP, and then we can re-enroll them…..
Killion wasn’t backing away this week.
He repeated the same advice Monday during a visit to West Side schools from Glenda Ritz, Indiana’s superintendent of public instruction. (Ritz didn’t jump on board, instead calling on parents get their kids ready for ISTEP days.)
And on Tuesday, Killion clarified the statement, saying he wasn’t necessarily advocating the withdraw/home-school/re-enroll plan.
“Since there’s no legislative mechanism, that’s the only opt-out workaround that I know to tell parents,” Killion said. “Typically, when I’m asked a question, I try to come up with the correct answer, and that’s what’s happened in this case.”
The journalist writing the column is critical of Killion and so is this legislator:
State Sen. Brandt Hershman, R-Buck Creek, wasn’t pleased to hear a superintendent “encouraging people to willfully thwart (the) system.”
“It’s just the latest episode in his series of irresponsible and provocative comments that bear little to no relevance to the school system he’s supposed to be leading,” Hershman said Tuesday, a day when the Senate was dealing with a bill that would strip some of Ritz’s authority and a resolution to shorten ISTEP that had doubled in length since last year.
“I think we test too much, and the ISTEP is not perfect, but testing is required under federal and state law,” Hershman said. “His comments represent a flawed example of leadership in education policy.”
Killion’s answer: “The only thing I’ve said is what I said in the interview when a reporter asked me how can parents opt out of ISTEP. That’s the only thing I’ve done.”
Martin Luther King, Jr., said: “One has a moral responsibility to disobey unjust laws.”
Welcome to the honor roll, Rocky Killion!
Diane Ravitch's blog 
I first heard this proposal from Senor Swacker.
Good insight, Duane. Your idea is catching on, it seems.
Thanks, Joanna.
Yes I have mentioned that option quite a number of times and most people look at me like I’m insane (they’re not far off). And no doubt that many others have thought the same thing but have not voiced the thought.
What is amazing is that Killion is proposing this from a position of authority (in the good sense) in that many more folks will hopefully start to take action (who listens to an old fart Spanish teacher?).
Great job opportunity in Fullerton California for a Superintendent who wants to work in one of the best districts in the nation. Please apply. We do not want to repeat the same mistakes of Florida. We are just beginning the testing process and we have a Governor who is on record of NOT BEING IN FAVOR OF NATIONAL TESTING.
Finally, a Superintendant who is willing to tell the truth about what is really happening to our schools. Instead of bullying parents into believing they have no options, he gives them a real alternative. It’s a shame that this is the only alternative available. The Feds have our public schools “caught between a rock and a hard place”. Instead of blaming this brave Superintendant and our teachers, let’s place the blame where it realy belongs, with our state and federal elected officials who have set us all up for failure. This obsession with standardized mandatory testing is emotional child abuse at its worst.
I don’t need a standardized test to tell me what we already know. Middle class and wealthy children are doing fine, and children in poverty continue to struggle. A standardized test will never close the gap caused by poverty. Only policies that address the cycle of poverty, such as access to jobs, being able to earn a living wage, stable housing in safe neighborhoods, and access food and health care, etc., can even begin to touch the tip of the iceberg. If we stop wasting public tax dollars on testing that tells us nothing, then we can fund the real resources needed to help educated our children.
Federal NCLB law requires that we use research based methods, yet there is NO research to support standardized testing as a tool to improve educational outcomes. When will we have real conversations about what we know the research does support? You know, the kinds of resources we see in the schools where children of those wealthy EduDeformers attend. Why don’t our wonderful philanthropists ( Gates, Walton, Broad, etc.) help fund lower class sizes in high poverty schools, instead of test and punish “reforms”?
Finally, parents are beginning to see the results of the harmful EduDeform initiatives that have taken over our schools. The mainstream media did a good job of covering up what we in education have been saying all along. Parents will do what they need to do to protect their children. We need to support them. We all have a stake in this. The future of our democracy depends on it.
“. . . yet there is NO research to support standardized testing as a tool to improve educational outcomes. When will we have real conversations about what we know the research does support?”
We know that the research supports the complete invalidities of the educational malpractices of educational standards and standarized testing. Wilson has shown the many epistemological, ontological, and logical errors involved in the process.. The conversation starts when we reject those malpractices (including the “grading” of children). Anything before that point is mental masturbation.
Thanks Duane, for sharing Wilson’s work with us. The problem is that our elected officials are clueless about education and are happy to support this race to nowhere. It’s time we hold those elected officials accountable for their complicity in passing these abusive laws, at both federal and state level. Yet teachers continue to vote into office people who do not support public education. As long as we continue to remain nonpolitical and voiceless in our classrooms, we will continue to remain complicit in our own demise.
Not all are clueless, Bridget. Many are just responding to the fact that privatizers fill their campaign coffers with corporate money. Facts, votes, and what’s good for students doesn’t matter…money does.
I’d be more selective about your use of the collective “we.” Educators cannot do their job without having a full and robust understanding of the strengths and weaknesses of their students. So without using assessment how is it you believe an educator would gather this information? If you teach at all, you assess your students at every possible opportunity (e.g., cold calling, pop quizzes, unit tests, etc…). The problem is when people misinterpret the point and usefulness of assessment. If anyone ever told you assessment was the cure for different achievement gaps they absolutely misspoke. What is does cure is quantifying exactly how large and/or small that gap is with the hope that those who are “educated” and “educating” children will actually make some attempt to use data to determine how to improve the support provided to children. It’s a shame that the district had technical difficulties and even a bigger shame that so many educators focus heavily on the policies that Dr Ravitch advocated for – before completely changing her mind that is. Accountability systems don’t help children, but if they provide the impetus for educators to collectively share their knowledge and understanding of the issues children are faced with, then it is an indirect route that I can live with.
Also, your statement about the “research” suggesting that assessment is completely invalid is ridiculous/laughable. What you’re saying is that an entire field of study (psychometrics) completely lacks an understanding of validity, reliability, and generalizability theories. Considering you didn’t attempt making an empirical case, I’ll assume that’s because there is no empirical evidence to support your position, just people who lack a fundamental understanding of quantitative science squawking about.
wbuchanan,
Are you clouding the issue—intentionally? Please correct me if I’m wrong, and then explain what you really mean, because I don’t understand what you are talking about.
I taught for thirty years (1975-2005) and assessment was always an ongoing daily part of the job before the insane era of standardized state and now federal tests (that masquerade as state generated tests) that have nothing to do with real assessment that is meant to improve instruction and learning for students who are cooperative and engaged.
For instance, a standardized bubble test doesn’t know the difference between students who study, do the work and ask questions and students that don’t. But when a teacher collects an essay and only half the class turns it in, what does the teacher do to get those children who didn’t do the essay to cooperate and work?
I’ll tell you what I was doing back in the late 1970s all the way up to August 2015. I was calling parents every night and telling them what the work was. I told those parents that I was in my classroom before school, at lunch and after school—and my students knew I was there available for them if they were having trouble with the work because I told them EVERY day. Guess what—99.9% of the teens that didn’t do the work also didn’t take advantage of those teacher office hours. The school also set up after school tutoring and counseled the students that needed that help the most to attend. Most of the students who weren’t cooperating also didn’t voluntarily attend tutoring.
Then the district offered summer school classes to those at risk students who were falling behind—not because of teaching but because most of them weren’t making any attempt to learn.
The first day of summer school—I taught summer school for at least half of the years I was in the classroom—would start with about 50 students in each class on the first day. Within a week, that number was less than twenty because the students who dropped out were bored with learning and wanted to be out on the street or hanging out at the nearest mall.
In addition, every assignment I corrected for the work that was turned in informed (that’s assessment) me as I was correcting late at night or on the weekends if the students were learning what I taught. If needed, the lesson would be taught again using a different approach. You see, I worked about 60 to 100 hours a week for most of those thirty years.
Assessment, as you might see, was an ongoing process that also included department meetings and grade level teams to assess student needs and discuss how to meet them through the curriculum we were teaching.
But how do teachers use the Common Core standardized tests when they never see the results. Tests that are not used for assessment but are used to rank teachers, fire teachers and close public schools—-tests that aren’t required in corporate Charters—do not play a roll in individual student assessment that is a daily ongoing job. Tests that ignore the challenges and results caused by living in poverty. Tests that ignore learning disability. Tests that ignore dysfunctional families. Tests that ignore the students who do not cooperate, study, ready, or do the work.
That isn’t assessment. That is punishment.
The corporate reform movement never brings up the issue of what it takes to learn. The entire focus of this fake reform movement based on rank and yang bubble tests is on teaching, and it ignores the fact that students must also be part of the process with support from parents and/or guardians.
wbuchanan,
The “we” was meant a tad facetiously. Yes, it is quite obvious that the vast majority of educators, and most everyone else, have no clue about Wilson’s work. I drew the short straw so it has fallen upon me to enlighten the world in the Quixotic quest to get folks to understand the COMPLETE INVALIDITIES involved in educational standards, standardized testing, and the nefarious “grading” of students.
“Educators cannot do their job without having a full and robust understanding of the strengths and weaknesses of their students. So without using assessment how is it you believe an educator would gather this information? If you teach at all, you assess your students at every possible opportunity (e.g., cold calling, pop quizzes, unit tests, etc…).”
Yes, educators can “do their job without having a ‘full and robust’ [whatever that means, sounds like edudeformerspeak to me] understanding of the . . . .” I’ve been teaching for 21 years now and have never even come close to, and have never tried to, having a “full and robust” understanding what a student has learned, is learning for that is a chimera, a duende, a false perception of what occurs in the teaching and learning process.
Do I assess students? Yes, every minute of my teaching day. And those assessments are fuzzy, subjective, many times out right wrong, and are never objective, precise and/or a “measurement” of what the students are thinking and learning. It most certainly does not “cure is quantifying exactly how large and/or small that gap”. The teaching and learning process can never be “quantified” in the manner in what you write.
“Pop quizzes”. Another educational malpractice that let’s the students know you don’t trust and respect them. I respect my students more than that. They’re as bad as those administrative walkthroughs that last 3-5 minutes. The two educational malpractices are means of false control of both students and teachers.
“Also, your statement about the “research” suggesting that assessment is completely invalid is ridiculous/laughable.”
Perhaps you need some Colemanian close reading strategies so that you may correctly cite what I wrote. I did not suggest that “assessment is completely invalid”. Go back and REREAD what I wrote.
“What you’re saying is that an entire field of study (phrenology*) completely lacks an understanding of validity, reliability, and generalizability theories.” I changed one word of your statement to help you see why charges of invalidity, unreliability and non-generalizability can be and/or are valid. (and by the way I didn’t say what you impute that I have said. If I did, Show Me (I’m from Missouri)
“Considering you didn’t attempt making an empirical case, I’ll assume that’s because there is no empirical evidence to support your position, just people who lack a fundamental understanding of quantitative science squawking about.”
The “empirical case” is not what is needed here. What is needed and what Wilson has done is to point out the errors of ontology and epistemology that saturate the whole educational standards and standardized testing discourse which renders those educational malpractices COMPLETELY INVALID!
wbuchanan, Have you read Wilson’s work? If so, please refute and/or rebut what he has proven. I’ve been looking for 15 years for a refutation/rebuttal and have yet to find one. Please if you have one then let me know, however, I am not holding my breath while awaiting a cogent response.
“. . . just people who lack a fundamental understanding of quantitative science squawking about.”
I’ll let PW Harris have the final say on that:
*Phrenology was a science of character divination, faculty psychology, theory of brain and what the 19th-century phrenologists called “the only true science of mind.”
wbuchanan,
I’ll put the Wilson link below for you.
Great job opportunity in Fullerton California for a Superintendent who wants to work in one of the best districts in the nation. Please apply. We do not want to repeat the same mistakes of Florida. We are just beginning the testing process and we have a Governor who is on record of NOT BEING IN FAVOR OF NATIONAL TESTING.
“It’s inhumane what we are doing to the kids, what we are doing to the educational environment, we lost so much instructional time today, it’s ridiculous,” Killion told WTHR-TV in Indianapolis on Feb. 12, after computers froze during a dry run for ISTEP last week.
If “It’s inhumane what we are doing to the kids”, is it not a form of child abuse?
Great question. Makes one wonder if basing teacher pay and school closures on a child’s performance aren’t a perversion of child labor laws.
Oops, …help educate our children…
I had the chance to listen to Rockey when he came to speak at our high school and viewed the movie they are talking about. Every parent with kids in public school should watch this movie it will open your eyes to the problems and unfairness in our schools. He is just answering the question he is asked honestly politicians don’t know what that means.
Rocky Killion, a man of courage and humanity.
“I think we test too much, and the ISTEP is not perfect, but testing is required under federal and state law,” Hershman said.
So he admits that the testing is a Federal mandate?
We see Hershman (R) use the augment “we can’t stop doing the wrong thing, just because what we have been doing is wrong”, Soon, we’ll see Republicans saying teachers don’t love America.
Reblogged this on McTeaching Ag and commented:
Thoughts from the 2015 Indiana Superintendent of the Year! Kudos to him.
Great job opportunity in Fullerton California for a Superintendent who wants to work in one of the best districts in the nation. Please apply. We do not want to repeat the same mistakes of Florida. We are just beginning the testing process and we have a Governor who is on record of NOT BEING IN FAVOR OF NATIONAL TESTING.
In Indiana, the great irony to ISTEP testing is this: Students are taught it is ethically wrong to cheat on a test that is cheating them out of a great education. When we know a student has cheated on ISTEP, the score is voided. The most ethical educational act Indiana could do is void the ISTEP entirely.
“but testing is required under federal and state law,” Hershman said. “His comments represent a flawed example of leadership in education policy.”
So, Hershman thinks we should all autocratically fall into line and obey all federal and state laws?
I don’t think so. There are many laws that are unjust.
For instance, the Chinese Exclusion Act (1882 – 1943) was the only/first federal law that legalized racism. The only/main reason the act was repealed in 1943 was becasue the Chinese were our allies in fighting Japan.
Indiana University, a public university, takes Koch money. And, it partnered with the plutocratic and corporate funded, NBER, Stanford and Harvard, for the National Center on School Choice.
It’d be ironic if Indiana K-12 employees, formed an organization, Center on Privatization of Indiana University. The nature of the faculty’s research, in response, would be interesting.
INDEED!!!!
For interested people, especially in Indiana.
Join the
Indiana Coalition for Public Education. [Best $25.00 you can spend right now.]
http://www.icpe@aol.com
or call: 317 752 1439
THEY may be our best hope.
Some superintendents in Indiana are banding together to fight
but
the powers that be “ain’t listening”
and
as mentioned here, even Glenda Ritz is not the answer.
Please consider joining in the fight. It MAY not be too late YET.
Go Rocky! Hey Hershman, ISTEP bears no relevance to the school system that Mr. Killion runs.
This post got me to thinking about times when I sat with my special ed students as they took a test. Just listening to them read the questions was eye opening as well as painful. The mistakes they made in word recognition that changed the whole question (sometimes into pure nonsense) made it clear why some of them eventually got readers for standardized testing. Then there were the kids who used so much mental energy getting through one page of a test who could no longer keep it together. I sat with a teacher as he was grading his papers. Initially he was so pleased with how one of his special ed students had done. I cautioned him to wait until he graded the second page. She/he totally blew it. The whole test was multiple choice, so it wasn’t due to a change in format. The student just needed the test over a few days. I had another student for whom I used to modify a regular teacher’s tests. The student could not deal with the two column matching exercises but could handle the same information as multiple choice. I shudder to think what these students would do with the PARCC tests. Their scores would probably suggest that they should be institutionalized. Balderdash! I want to hit something every time some talking head says the tests will inform instruction.
As a child starting school in the early 1950s, I had dyslexia so bad, that after some so called experts gave me a test of some kind gave me a test that I don’t remember taking, my mother was told I would never learn to read or write. And even though my mother made liars out of those experts and she taught me to read at home, I still have dyslexia and because of it, when I’m reading it is always possible for me to see a word that isn’t on the page that I’m reading because during the translation process of seeing and sending what is seen on the page to the brain, the message might get scrambled and end up in the brain as something else changing the context of the sentence and causing a child with dyslexia to select the wrong answer in a computerized bubble test.
How many children have dyslexia? According to the BBC, about 1 in 10—in the U.S. that could mean about 5 million school age children.
http://news.bbc.co.uk/cbbcnews/hi/find_out/guides/tech/dyslexia/newsid_1747000/1747150.stm
But according to Yale, dyslexia affects 1 out of 5 people—63 million Americans or 10 million school children.
Here’s what Yale has to say about dyslexia: “While there are numerous curricula and programs designed to increase literacy, dyslexia is often overlooked when searching for causes of illiteracy. Dyslexia is the most common reading disability—20% of the population is struggling with this hidden disability, and many remain undiagnosed, untreated, and struggling with the impact of their dyslexia. The diagnosis and treatment remain elusive in public schools, and even more so in urban school populations, and African American and Latino communities.”
http://dyslexia.yale.edu/MDAI/
Common Core and the Bill Gates driven/supported rank and yank insanity behind the standardized testing agenda does absolutely nothing to deal with dyslexia, and teachers who are judged by their students’ scores are victims of this Common Core Crap.
I know what you mean. Here in Louisiana the second tier modified alternate assessment that many of our sped kids have been taking for years has suddenly been removed. These kids now have to take the same assessment as every other student. They are, by definition of their sped exceptionality, functioning with reading deficits and learning disabilities. As you have stated, reading at grade level is grueling and takes so much mental effort that their reading comprehension suffers. Most will give up before they can finish the test, even with extended time and other accommodations. I can’t wait to see the results, when they finally come out next School year. Of course the blame will be placed on teachers, not on the abusive testing practices.
think we test too much, and the ISTEP is not perfect, but testing is required under federal and state law,” Hershman said. “His comments represent a flawed example of leadership in education policy.”
The only way the federal government can require ANYTHING from education is if states accept federal monies for education. Education is a function of the STATES not the Federal government. If states would quit chasing Federal $$ of education, they could do away with ISTEP.
This one’s for you wbuchanan!! Download the dissertation, read and enjoy, and begin to understand why educational standards and standardized testing are indeed COMPLETELY INVALID educational malpractices:
“Educational Standards and the Problem of Error” found at: http://epaa.asu.edu/ojs/article/view/577/700
Brief outline of Wilson’s “Educational Standards and the Problem of Error” and some comments of mine.
1. A description of a quality can only be partially quantified. Quantity is almost always a very small aspect of quality. It is illogical to judge/assess a whole category only by a part of the whole. The assessment is, by definition, lacking in the sense that “assessments are always of multidimensional qualities. To quantify them as unidimensional quantities (numbers or grades) is to perpetuate a fundamental logical error” (per Wilson). The teaching and learning process falls in the logical realm of aesthetics/qualities of human interactions. In attempting to quantify educational standards and standardized testing the descriptive information about said interactions is inadequate, insufficient and inferior to the point of invalidity and unacceptability.
2. A major epistemological mistake is that we attach, with great importance, the “score” of the student, not only onto the student but also, by extension, the teacher, school and district. Any description of a testing event is only a description of an interaction, that of the student and the testing device at a given time and place. The only correct logical thing that we can attempt to do is to describe that interaction (how accurately or not is a whole other story). That description cannot, by logical thought, be “assigned/attached” to the student as it cannot be a description of the student but the interaction. And this error is probably one of the most egregious “errors” that occur with standardized testing (and even the “grading” of students by a teacher).
3. Wilson identifies four “frames of reference” each with distinct assumptions (epistemological basis) about the assessment process from which the “assessor” views the interactions of the teaching and learning process: the Judge (think college professor who “knows” the students capabilities and grades them accordingly), the General Frame-think standardized testing that claims to have a “scientific” basis, the Specific Frame-think of learning by objective like computer based learning, getting a correct answer before moving on to the next screen, and the Responsive Frame-think of an apprenticeship in a trade or a medical residency program where the learner interacts with the “teacher” with constant feedback. Each category has its own sources of error and more error in the process is caused when the assessor confuses and conflates the categories.
4. Wilson elucidates the notion of “error”: “Error is predicated on a notion of perfection; to allocate error is to imply what is without error; to know error it is necessary to determine what is true. And what is true is determined by what we define as true, theoretically by the assumptions of our epistemology, practically by the events and non-events, the discourses and silences, the world of surfaces and their interactions and interpretations; in short, the practices that permeate the field. . . Error is the uncertainty dimension of the statement; error is the band within which chaos reigns, in which anything can happen. Error comprises all of those eventful circumstances which make the assessment statement less than perfectly precise, the measure less than perfectly accurate, the rank order less than perfectly stable, the standard and its measurement less than absolute, and the communication of its truth less than impeccable.”
In other word all the logical errors involved in the process render any conclusions invalid.
5. The test makers/psychometricians, through all sorts of mathematical machinations attempt to “prove” that these tests (based on standards) are valid-errorless or supposedly at least with minimal error [they aren’t]. Wilson turns the concept of validity on its head and focuses on just how invalid the machinations and the test and results are. He is an advocate for the test taker not the test maker. In doing so he identifies thirteen sources of “error”, any one of which renders the test making/giving/disseminating of results invalid. And a basic logical premise is that once something is shown to be invalid it is just that, invalid, and no amount of “fudging” by the psychometricians/test makers can alleviate that invalidity.
6. Having shown the invalidity, and therefore the unreliability, of the whole process Wilson concludes, rightly so, that any result/information gleaned from the process is “vain and illusory”. In other words start with an invalidity, end with an invalidity (except by sheer chance every once in a while, like a blind and anosmic squirrel who finds the occasional acorn, a result may be “true”) or to put in more mundane terms crap in-crap out.
7. And so what does this all mean? I’ll let Wilson have the second to last word: “So what does a test measure in our world? It measures what the person with the power to pay for the test says it measures. And the person who sets the test will name the test what the person who pays for the test wants the test to be named.”
In other words it attempts to measure “’something’ and we can specify some of the ‘errors’ in that ‘something’ but still don’t know [precisely] what the ‘something’ is.” The whole process harms many students as the social rewards for some are not available to others who “don’t make the grade (sic)” Should American public education have the function of sorting and separating students so that some may receive greater benefits than others, especially considering that the sorting and separating devices, educational standards and standardized testing, are so flawed not only in concept but in execution?
My answer is NO!!!!!
One final note with Wilson channeling Foucault and his concept of subjectivization:
“So the mark [grade/test score] becomes part of the story about yourself and with sufficient repetitions becomes true: true because those who know, those in authority, say it is true; true because the society in which you live legitimates this authority; true because your cultural habitus makes it difficult for you to perceive, conceive and integrate those aspects of your experience that contradict the story; true because in acting out your story, which now includes the mark and its meaning, the social truth that created it is confirmed; true because if your mark is high you are consistently rewarded, so that your voice becomes a voice of authority in the power-knowledge discourses that reproduce the structure that helped to produce you; true because if your mark is low your voice becomes muted and confirms your lower position in the social hierarchy; true finally because that success or failure confirms that mark that implicitly predicted the now self evident consequences. And so the circle is complete.”
In other words students “internalize” what those “marks” (grades/test scores) mean, and since the vast majority of the students have not developed the mental skills to counteract what the “authorities” say, they accept as “natural and normal” that “story/description” of them. Although paradoxical in a sense, the “I’m an “A” student” is almost as harmful as “I’m an ‘F’ student” in hindering students becoming independent, critical and free thinkers. And having independent, critical and free thinkers is a threat to the current socio-economic structure of society.
By Duane E. Swacker
I’ve never actually said I agree with categorizing children and/or dichotomizing/categorizing data at all. Anyone with a rudimentary understanding of quantitative analysis/statistics would tell you that it is a horrible idea and does nothing but throw away valuable information when giant bins are created out of continuous data (since learning as I hope you would agree is a continuum).
So, in response to your constant reliance on this single piece of literature as the crux of all that should be the end of assessment and measurement, here are some responses in the few minutes that I’ve dedicated to reading what seems to be fairly misguided “research”:
“Most studies of error in the measurement of standards are however much more
specific in their focus than is mine” (p. 2).
We don’t measure standards. We measure the skills, abilities, and/or aptitudes of the examinee. The standards are used to define what skills, abilities, and/or aptitudes the examinee should possess. Standards are created by a combination of policy makers and educators, not psychometricians.
“by detailing some of the psychometric fudges on which many assessment
claims depend to maintain their established meaning.” (p. 8)
All of the underlying assumptions of psychometric models are very explicit. The “fudges” are really issues with people who lack an understanding of the mathematics behind the assumptions and models.
Since I have other things to do today, this is all I’ll comment on for now. But why not try to present evidence from dissenters within the field who can debate and articulate the deficiencies in the math rather than relying solely on philosophical arguments which can easily be justified in either direction?
Wbuchanan, as you know, standard-setting is not scientific. It is based on subjective judgement. There is no science to determine what children should know and do at any grade level. It depends… On many factors.
wbuchanan,
Thanks for the responses as these dialogues help me sharpen my arguments and learning. If I may ask, what is your relation to public education?
I’d like to start with your last question. Wilson IS a dissenter from within the field:
“My last year in the RAAF was spent in the trade testing section. . . Because of my experience with multiple choice tests in the RAAF, I had been working with Australian Council for Educational Research on the construction of multiple choice physics tests. When a full-time position came up I applied for it. For the next six years I was to work as a test constructor. I learnt a lot about the nature and mechanics and rituals of testing, about the truisms and tricks of the trade. For example, that only “items” between thirty and seventy percent difficulty were chosen because others did not contribute economically to the separation of students; that seemingly almost identical questions often had very different difficulty levels; and it was almost impossible to tell, without prior testing, how difficult a test item was. Central to the theme of this study, I also learnt, at the level of practice and praxis, the great secret about error, about the fallibility of the human judge, about the vagueness and arbitrariness of the standard. Not in that language, of course. Psychometrics provides a more prophylactic discourse about marker reliability and predictive validity and generalizability. Even so, it was impossible to miss the point.. . . When at age forty I was appointed to head the newly established Research and Planning Branch in the SA Education Department, a position I held (with planning dropped half way through), for the next thirteen years, my major claim to expertise was in the area of testing and assessment.”
“. . . here are some responses in the few minutes that I’ve dedicated to reading what seems to be fairly misguided “research””. Let me add (from before and after the quote) to the thought you quote from Wilson:
“On the other hand, most of the literature on reliability and validity is pertinent to this study, because, when its discourse is repositioned from examiner to examined, it provides more than enough invalidity information to self destruct. Most studies of error in the measurement of standards are however much more specific in their focus than is mine. Their minimal effect on practice has perhaps partially been due to the fact that their critiques were in terms of their own discipline of educational measurement; a discipline [psycometrics] that owes its very existence to the CLAIM TO ACCURATE JUDGMENTS.” (MY EMPHASIS)
And from you: “We don’t measure standards. We measure the skills, abilities, and/or aptitudes of the examinee. The standards are used to define what skills, abilities, and/or aptitudes the examinee should possess.”
By definition a standard has to be measured before it can become a standard. And that is part of the problem that Wilson points out; the different meanings of “standard” and how the confusing and conflating of the different meaning obscure the discourse. And although “the standards” supposedly are used to define skills, abilities and/or aptitudes, Wilson proves how that cannot be the truth of the matter.
To bring the discussion back around to your last question about “rather than relying solely on philosophical arguments which can easily be justified in either direction” (and no they can’t be “justified in either direction”-which Wilson proves in his study). What the above paragraph hints at is problems in definitions, usages, meanings, error, etc. . . that demand that fundamental epistemological and ontological assumptions that are the foundations of psychometrics be examined more closely and that the way to do that is not through empirical studies but through logical philosophical arguments as Wilson has done.
If “relying on philosophical arguments which can easily be justified in either direction” were to be true in this case, then you, or anyone should be easily able to refute/rebut what Wilson has shown to be COMPLETELY INVALID educational malpractices that cause much harm to many students and now through related malpractices (VAM, SGP, etc. . . ) to teachers, schools and districts.
Please read the whole study. And as I tell my students when I have them complete a test or quiz: “Have at it and have fun!!!” (when they first hear that they think I’m nuts but as time goes on they come to understand what I am trying to say). If you have any questions/comments please feel free to email me at dswacker@centurytel.net and please reference Wilson’s work in the subject line.
Take care, hopefully where you’re at it is not quite as bone chillingly cold as it is here in the beautiful Missouri River hills of southern Warren County, MO.
I am honored to be named to Dr. Ravitch’s honor roll. Also, I’m deeply grateful to her as well as to Peter Coyote, Dr. Linda Darling-Hammond, Dr. Pasi Sahlberg, Dr. Marc Tucker, Jamie Vollmer, and the many Indiana teachers and administrators who helped make “Rise Above the Mark” a reality. Directed by Jack Klink, script written by Angie Klink, and produced by Steve Klink and me, “Rise Above the Mark” is making a difference for public school students and teachers by educating the general public about how politicians are dismantling public schools, and what we all can do to save our public schools.
Is this the proper link?
http://www.amazon.com/Rise-Above-Mark-Diane-Ravitch/dp/B00N86DAXW/ref=sr_1_1?s=movies-tv&ie=UTF8&qid=1424752601&sr=1-1&keywords=Rise+ABove+the+Mark
“For Rise Above the Mark”, do you have any idea what it takes to get an Amazon sellers rank like this one—-Amazon sells literally millions of products (for books alone, there are more than 12 million titles)
Amazon Best Sellers Rank: #14,198
They must be flying off the shelf.