John Richard Schrock teaches science to future science teachers in Kansas. This is his view of what is needed to improve assessment.
He writes:
President Obama’s proposal to cap external assessments at two percent of student class time is seven years late and two percent too much. It does not end the educational disaster of 14 years of No Child Left Behind (NCLB) over-testing. It does not bring back the art and music classes that were lost because they were not tested and therefore did not count. Nor does it address the concerns of growing number of parents who are opting their child out of testing. And it does nothing to re-professionalize teaching.
Every rural Kansan knows that the more time you spend weighing them, the less time you have to feed them. But reducing testing to two percent does not mean that a teacher will have 98 percent of class time for teaching. While the last 14 years of assessments only consumed a week each spring, the months before the test were often filled with pre-tests, practicing for the tests, and every form of coercion imaginable to get students to score higher. With teachers and administrators still under-the-gun to raise test scores, this teaching-to-the-test will continue. Indeed, in most states the current mandated assessments only take up 2.7 percent of class time. But preparation for that test consumes the months beforehand. Reducing the actual testing to two percent of class time does nothing to eliminate the test-prep.
To weigh the effect of NCLB on the teaching profession, consider what it would do to the medical profession if this standardization was imposed on doctors. Previously, physicians treated each patient who came in with unique needs and left with individualized cures. And teachers taught students who came in unique and left unique.
But teachers are restricted to scores on language arts and math. That is like forcing doctors to only use temperature and blood pressure to rate a patient’s health. As a result, patients get no attention to lung and kidney and other problems. And students are shortchanged in art, music, science and social studies.
With temperature and blood pressure the only indicator of health, and heavy penalties on doctors and hospitals that don’t improve those measures, physicians would load their patients up on aspirin and blood pressure medicine. Similarly, teachers have to teach-to-the-past-tests and raise assessment scores. Of course, the overall effect is sicker patients. And despite increased assessment scores, the genuine measurements of student abilities on the NAEP, SAT and ACT go down.
The ACT and SAT have been around far longer than the NCLB testing mania. So why weren’t they just as bad as current assessments? The ACT and old SAT are aptitude tests, not achievement tests. They measured a students aptitude or general ability. Generally, a teacher cannot teach-to-the ACT or SAT tests, so it did not distort their classroom teaching. These tests do not promote memorization and drillwork.
But the government-mandated assessment tests are achievement tests that do respond to memorization and drillwork. State boards of education latch onto standards that profess fanciful creative-thinking goals. But teachers under pressure don’t teach-to-the-standards; they await the release of the first round of tests and they teach-to-that-test.
To treat patients as unique patients, physicians must have the total professional judgement call on what tests to use—period.
And to treat our students as the unique students they are, teachers must regain their professional right to be the sole testers of their students. There should be no external test that requires them to teach-to-that-test. Not two percent, Mr. President. Zero percent.
Ivory tower educationists rail that math and English are universal across the U.S. and therefore the tests must be universal. But teaching is about students as much as about the subject. City kids do not have the same experience base as rural students.
American teachers were unique in the world because we had the professional right and responsibility to teach different students differently. To restore our profession, we must regain that right. Our students come to us unique; they should leave our classrooms unique.
No more standardization means no more external testing.

“The ACT and SAT have been around far longer than the NCLB testing mania. So why weren’t they just as bad as current assessments? The ACT and old SAT are aptitude tests, not achievement tests. They measured a students aptitude or general ability. Generally, a teacher cannot teach-to-the ACT or SAT tests, so it did not distort their classroom teaching. These tests do not promote memorization and drillwork.”
Uh, no, not really. There really isn’t a way to “measure” (or even assess) aptitude or general ability. Both SAT (old and new) and ACT are most definitely subject to teaching to the test through memorization and drillwork – that’s why there are very lucrative industries devoted to doing just that.
The difference is that teachers and schools have never been held “accountable” for SAT or ACT scores, so right there you lose the pressure to teach to them. If there are stakes attached to tests that affect teachers’ compensation or even jobs, or the future of the schools, there will be drillwork to prep for the tests no matter what kind of tests they are.
Otherwise, excellent article and right on point.
LikeLike
Agreed.
😎
LikeLike
Beat me to the punch on that portion, Dienne. Had it highlighted and copied, ready to go. Totally agree with your comment on that quote.
But I’m not sure that it is an “excellent” article.
Right before that quote Shrock states: “the genuine measurements of student abilities on the NAEP, SAT and ACT go down.” I have found that math and science teachers are the ones who are the hardest to convince that standardized test scores are complete hogwash. It seems because they work with “true”, “real” and “valid” information of math and science that everything can be “measured” as if in an experiment. As a generalization, math and science teachers seem to lack an understanding of what the enlightenment principles as embodied in a scientific mindset entail. It’s as if they live/work in a field in which the “truths” are everlasting and eternal and there is no need to question them.
These teachers (and it includes many other subject/grade level teachers) do not understand the underlying concept of the scientific mindset–fidelity to truth. Fidelity to truth means that when your “truth” is shown to have error, falsehoods and/or contradictions then you must change your “truth” to accommodate that information or completely abandon and reject that “truth”.
Shrock’s “truth” of “genuine measurements of student abilities” has been shown by Noel Wilson in his never refuted nor rebutted 1997 dissertation to contain, to be based on epistemological and ontological errors, falsehoods and contradictions rendering said assessments COMPLETELY INVALID.
LikeLike
Are you trying to say that I honestly can’t measure the multiplication skills of my math students with some degree of accuracy? Tests certainly can’t accurately measure everything but to constantly reiterate that tests can measure nothing seems equally ridiculous. Maybe I am not grasping the Wilson belief system.
If I teach students how and why we balance chemical equations, are you suggesting that this skill and understanding cannot be tested? I would beg to differ. Some students can successfully balance equations but can also explain why in a very adolescently articulate way. Would such a test really yield zero information to the chemistry teacher?
LikeLike
Rage,
Who would do a better job of testing what you taught? You or Pearson?
LikeLike
Yes, I am saying that you can’t honestly measure any part of the teaching and learning process. You surely can assess parts, you can make judgements on parts of the teaching and learning process but you can’t measure any part.
One can’t measure because there is no agreed upon standard of measurement (construct validity issues), no agreed upon measuring device with no gauge to determine whether the measuring device is accurate. Please point to those crucial components of any supposed educational measuring device. I know of none.
You can count correct answers but counting is not measuring. And no, a teacher made test wouldn’t yield zero information, it could yield plenty of information, none of which is a measurement of the student’s learning but can be an assessment of the student’s learning.
And no, it’s not all semantics.
The usage of the terms dealing with measuring/measurement imply a certain supposed scientific and logical certainty in the assessment that is not there. And those implications of pseudo-accuracy, pseudo-validity and pseudo-reliability of testing in the process of teaching and learning have far reaching consequences not only for the students, but teachers, schools, etc. . . in purporting to be “neutral” or non-biased assessments.
Have you read Wilson’s work (not my Cliff/Spark Note version)??? If so, do you have any questions about what he has demonstrated to be error filled and invalid standards and tests?
LikeLike
Rage – the issue is with the word “measure”. You can only measure “amounts” of something – how many inches long, how many pounds heavy, etc. There is no “amount” of learning – it doesn’t come in incremental bits that can be objectively seen or compared against some agreed-upon standard. “Test” or “assess” are better words when it comes to trying to figure out how much a student has learned, so long as it’s understood that there is no precise way to determine some ultimate truth.
LikeLiked by 1 person
Well said, Dienne! Muchas Gracias.
LikeLike
Diane
I’ve been raging against the Pearson testocracy here for years. I am 100% against standardized testing. Even our NY Regents tests have turned classes into year long test prep experiences. Can’t tell you how many Regents teachers tell me that, “we don’t teach it if its not on the Regents test”. Same with AP courses.
Dienne
Sounds kind of nit-picky, but I think I get what you’re saying. Despite Duane’s comment, it still seems more like a semantics argument. So if a math student scores 40 out 50 multiplication calculations, the test did not “measure” their ability at 80% but fairly “assessed” their multiplication skills (on that day) at 80% proficient. I wonder how a psychometrician would respond to this? These highly trained experts seem to think that they can measure skills and knowledge.
From Wikipedia:
Psychometrics is a field of study concerned with the theory and technique of psychological measurement. One part of the field is concerned with the objective measurement of skills and knowledge, abilities, attitudes, personality traits, and educational achievement. For example, some psychometric researchers have, thus far, concerned themselves with the construction and validation of assessment instruments such as questionnaires, tests, raters’ judgments, and personality tests. Another part of the field is concerned with statistical research bearing on measurement theory (e.g., item response theory; intraclass correlation).
As a result of these focuses, psychometric research involves two major tasks: (i) the construction of instruments; and (ii) the development of procedures for measurement. Practitioners are described as psychometricians. Psychometricians usually possess a specific qualification, and most are psychologists with advanced graduate training. In addition to traditional academic institutions, many psychometricians work for the government or in human resources departments. Others specialize as learning and development professionals.
LikeLike
Rage,
The semantics being very important for the reason I mentioned: one of “measuring” lending a scientific veneer, a pseudo-objective sheen to a process that doesn’t warrant/deserve that significance/connotation/association. In the sense that “measuring” is a false signification (which I believe is intentional also) it should be rejected and its usage challenged every time.
Psychometrics is a modern day amalgam of phrenology and eugenics. There are no direct measurements of anything that occurs in the grey matter of living breathing humans, only secondary, tertiary and semi-related phenomena that some (psychometricians) like to brandish as being legitimate scientific inquiry. It’s not!
LikeLike
“These highly trained experts seem to think that they can measure skills and knowledge.”
And they are thinking wrong if that is what they proclaim. Many groups over the course of intellectual history (in other words the history of mankind) have believed that they held the key to understanding the human mind only to have those keys break apart when attempting to open the lock on the human mind and thinking. Need I give the many examples? Psychometrics is just the current version of the wrong key.
LikeLike
Oops, hit post comment to soon. To understand those COMPLETE INVALIDITIES, I invite all to read and comprehend Wilson’s “Educational Standards and the Problem of Error” found at: http://epaa.asu.edu/ojs/article/view/577/700
Brief outline of Wilson’s “Educational Standards and the Problem of Error” and some comments of mine.
1. A description of a quality can only be partially quantified. Quantity is almost always a very small aspect of quality. It is illogical to judge/assess a whole category only by a part of the whole. The assessment is, by definition, lacking in the sense that “assessments are always of multidimensional qualities. To quantify them as unidimensional quantities (numbers or grades) is to perpetuate a fundamental logical error” (per Wilson). The teaching and learning process falls in the logical realm of aesthetics/qualities of human interactions. In attempting to quantify educational standards and standardized testing the descriptive information about said interactions is inadequate, insufficient and inferior to the point of invalidity and unacceptability.
2. A major epistemological mistake is that we attach, with great importance, the “score” of the student, not only onto the student but also, by extension, the teacher, school and district. Any description of a testing event is only a description of an interaction, that of the student and the testing device at a given time and place. The only correct logical thing that we can attempt to do is to describe that interaction (how accurately or not is a whole other story). That description cannot, by logical thought, be “assigned/attached” to the student as it cannot be a description of the student but the interaction. And this error is probably one of the most egregious “errors” that occur with standardized testing (and even the “grading” of students by a teacher).
3. Wilson identifies four “frames of reference” each with distinct assumptions (epistemological basis) about the assessment process from which the “assessor” views the interactions of the teaching and learning process: the Judge (think college professor who “knows” the students capabilities and grades them accordingly), the General Frame-think standardized testing that claims to have a “scientific” basis, the Specific Frame-think of learning by objective like computer based learning, getting a correct answer before moving on to the next screen, and the Responsive Frame-think of an apprenticeship in a trade or a medical residency program where the learner interacts with the “teacher” with constant feedback. Each category has its own sources of error and more error in the process is caused when the assessor confuses and conflates the categories.
4. Wilson elucidates the notion of “error”: “Error is predicated on a notion of perfection; to allocate error is to imply what is without error; to know error it is necessary to determine what is true. And what is true is determined by what we define as true, theoretically by the assumptions of our epistemology, practically by the events and non-events, the discourses and silences, the world of surfaces and their interactions and interpretations; in short, the practices that permeate the field. . . Error is the uncertainty dimension of the statement; error is the band within which chaos reigns, in which anything can happen. Error comprises all of those eventful circumstances which make the assessment statement less than perfectly precise, the measure less than perfectly accurate, the rank order less than perfectly stable, the standard and its measurement less than absolute, and the communication of its truth less than impeccable.”
In other words all the logical errors involved in the process render any conclusions invalid.
5. The test makers/psychometricians, through all sorts of mathematical machinations attempt to “prove” that these tests (based on standards) are valid-errorless or supposedly at least with minimal error [they aren’t]. Wilson turns the concept of validity on its head and focuses on just how invalid the machinations and the test and results are. He is an advocate for the test taker not the test maker. In doing so he identifies thirteen sources of “error”, any one of which renders the test making/giving/disseminating of results invalid. And a basic logical premise is that once something is shown to be invalid it is just that, invalid, and no amount of “fudging” by the psychometricians/test makers can alleviate that invalidity.
6. Having shown the invalidity, and therefore the unreliability, of the whole process Wilson concludes, rightly so, that any result/information gleaned from the process is “vain and illusory”. In other words start with an invalidity, end with an invalidity (except by sheer chance every once in a while, like a blind and anosmic squirrel who finds the occasional acorn, a result may be “true”) or to put in more mundane terms crap in-crap out.
7. And so what does this all mean? I’ll let Wilson have the second to last word: “So what does a test measure in our world? It measures what the person with the power to pay for the test says it measures. And the person who sets the test will name the test what the person who pays for the test wants the test to be named.”
In other words it attempts to measure “’something’ and we can specify some of the ‘errors’ in that ‘something’ but still don’t know [precisely] what the ‘something’ is.” The whole process harms many students as the social rewards for some are not available to others who “don’t make the grade (sic)” Should American public education have the function of sorting and separating students so that some may receive greater benefits than others, especially considering that the sorting and separating devices, educational standards and standardized testing, are so flawed not only in concept but in execution?
My answer is NO!!!!!
One final note with Wilson channeling Foucault and his concept of subjectivization:
“So the mark [grade/test score] becomes part of the story about yourself and with sufficient repetitions becomes true: true because those who know, those in authority, say it is true; true because the society in which you live legitimates this authority; true because your cultural habitus makes it difficult for you to perceive, conceive and integrate those aspects of your experience that contradict the story; true because in acting out your story, which now includes the mark and its meaning, the social truth that created it is confirmed; true because if your mark is high you are consistently rewarded, so that your voice becomes a voice of authority in the power-knowledge discourses that reproduce the structure that helped to produce you; true because if your mark is low your voice becomes muted and confirms your lower position in the social hierarchy; true finally because that success or failure confirms that mark that implicitly predicted the now self-evident consequences. And so the circle is complete.”
In other words students “internalize” what those “marks” (grades/test scores) mean, and since the vast majority of the students have not developed the mental skills to counteract what the “authorities” say, they accept as “natural and normal” that “story/description” of them. Although paradoxical in a sense, the “I’m an “A” student” is almost as harmful as “I’m an ‘F’ student” in hindering students becoming independent, critical and free thinkers. And having independent, critical and free thinkers is a threat to the current socio-economic structure of society.
LikeLike
Unlike other countries, teachers in this country are not very well-respected.
Embarrassing real story but this has to be told.
I actually have a cousin, with a college degree from Purdue, who made it quite clear to her mother and me that she has low regard for teachers. She was totally serious when she told her mother and me: “Well, my high school classmates think that IF YOU CAN’T DO ANYTHING, then you become a teacher.”
HUH? OMG!
She had a great public school education and graduated from Purdue. She was not impoverished. Father graduated from Purdue and Harvard; mother a stay-at-home mother.
My response after I caught my breath was: “You are doing what you love, you graduated from the Hawai’i Public Schools, you graduated from Purdue … and you don’t respect teachers? If I remember, you loved going to school and did well in school. Your parents weren’t divorced, you had a basically traumatic free childhood, teen life, and a great young adult life. I don’t understand why you think badly about the great teachers you had.
BTW …
Her father graduated from Purdue and Harvard. Mother a stay-at-home mother who was ACTIVE in the schools.
This BLAME public school teachers is another MINDLESS MANTRA in this country. Problem is our young are the “food” and the deformers expect us to serve our students those who own everything.
Don’t be fooled. We are still a racist, sexist, and classist society. Slavery is well and alive, only in a different form.
Scary. I don’t GET IT! 😦
Last thought: authoritarian and totalitarian are not the same. Question: What is our country over the past 30 years? Makes me shudder. And why? $$$$$$$$$$$$ and power — both illusions for which people have killed others and continue to kill. What a waste.
LikeLike
The testing craze has also infected medicine, at least in the US.
Many doctors have actually become obsessed with testing to the point where they completely disregard their patient’s description of symptoms.
Interestingly, it is often profit-making that drives tests both in the case of medicine and education.
LikeLike