Imagine this great victory for teachers in New York: They will now be allowed to discuss test questions that have been released to the public!
Is this progress? No. Suppose teachers spot unreleased questions that are clearly wrong, poorly worded, confusing, incoherent. If they have not been released to the public, the teachers are not allowed to criticize them or call attention to errors.
Peter Greene wondered if the New York Times recognized the absurdity of its headline, which claimed that the state was going to “relax” the gag order.
He wrote:
See, now the state will allow teachers to discuss items on the test after they have been publicly released, whereas previously, teachers could only discuss test items after they had been publicly released.
The gag order protects Pearson. If the gag order prevailed, we would have never known about the nutty question on a Pearson test about “the pineapple and the hare.” That question was not publicly released. It became public not because of teachers but because students complained about it, and it leaked to the New York City Parent blog.
This “gag order” is insulting to teachers. It should be eliminated. Its only purpose is to protect the interests of the testing companies. They should release all their questions. No one will know which will be on future tests. If there are thousands of test questions available, students can use them to see what is expected of them. And if they are released, parents and teachers will have a chance to evaluate their quality. That may be what scares the testing companies most.
Take off the gag!
Diane Ravitch's blog 
The quote from Peter Greene is the perfect summation of deformer logic.
LikeLike
From Peter: “It takes years of trials, not to mention money, to come up with questions appropriate for mass testing.
As always, test proctors are forbidden to look at the tests they are proctoring.”
For the first statement, there are no questions that are appropriate for mass testing as that “mass” testing is bogus, COMPLETELY INVALID epistemologically and ontologically to begin with.
Any certified teacher that gives a test (proctors it in standardized testing parlance) without reading said test is acting UNETHICALLY and should be held accountable/sanctioned for practicing unethically.
LikeLike
To understand that COMPLETE INVALIDITY read Noel Wilson’s never refuted nor rebutted complete takedown of educational standards and standardized testing:
“Educational Standards and the Problem of Error” found at: http://epaa.asu.edu/ojs/article/view/577/700
Brief outline of Wilson’s “Educational Standards and the Problem of Error” and some comments of mine.
1. A description of a quality can only be partially quantified. Quantity is almost always a very small aspect of quality. It is illogical to judge/assess a whole category only by a part of the whole. The assessment is, by definition, lacking in the sense that “assessments are always of multidimensional qualities. To quantify them as unidimensional quantities (numbers or grades) is to perpetuate a fundamental logical error” (per Wilson). The teaching and learning process falls in the logical realm of aesthetics/qualities of human interactions. In attempting to quantify educational standards and standardized testing the descriptive information about said interactions is inadequate, insufficient and inferior to the point of invalidity and unacceptability.
2. A major epistemological mistake is that we attach, with great importance, the “score” of the student, not only onto the student but also, by extension, the teacher, school and district. Any description of a testing event is only a description of an interaction, that of the student and the testing device at a given time and place. The only correct logical thing that we can attempt to do is to describe that interaction (how accurately or not is a whole other story). That description cannot, by logical thought, be “assigned/attached” to the student as it cannot be a description of the student but the interaction. And this error is probably one of the most egregious “errors” that occur with standardized testing (and even the “grading” of students by a teacher).
3. Wilson identifies four “frames of reference” each with distinct assumptions (epistemological basis) about the assessment process from which the “assessor” views the interactions of the teaching and learning process: the Judge (think college professor who “knows” the students capabilities and grades them accordingly), the General Frame-think standardized testing that claims to have a “scientific” basis, the Specific Frame-think of learning by objective like computer based learning, getting a correct answer before moving on to the next screen, and the Responsive Frame-think of an apprenticeship in a trade or a medical residency program where the learner interacts with the “teacher” with constant feedback. Each category has its own sources of error and more error in the process is caused when the assessor confuses and conflates the categories.
4. Wilson elucidates the notion of “error”: “Error is predicated on a notion of perfection; to allocate error is to imply what is without error; to know error it is necessary to determine what is true. And what is true is determined by what we define as true, theoretically by the assumptions of our epistemology, practically by the events and non-events, the discourses and silences, the world of surfaces and their interactions and interpretations; in short, the practices that permeate the field. . . Error is the uncertainty dimension of the statement; error is the band within which chaos reigns, in which anything can happen. Error comprises all of those eventful circumstances which make the assessment statement less than perfectly precise, the measure less than perfectly accurate, the rank order less than perfectly stable, the standard and its measurement less than absolute, and the communication of its truth less than impeccable.”
In other word all the logical errors involved in the process render any conclusions invalid.
5. The test makers/psychometricians, through all sorts of mathematical machinations attempt to “prove” that these tests (based on standards) are valid-errorless or supposedly at least with minimal error [they aren’t]. Wilson turns the concept of validity on its head and focuses on just how invalid the machinations and the test and results are. He is an advocate for the test taker not the test maker. In doing so he identifies thirteen sources of “error”, any one of which renders the test making/giving/disseminating of results invalid. And a basic logical premise is that once something is shown to be invalid it is just that, invalid, and no amount of “fudging” by the psychometricians/test makers can alleviate that invalidity.
6. Having shown the invalidity, and therefore the unreliability, of the whole process Wilson concludes, rightly so, that any result/information gleaned from the process is “vain and illusory”. In other words start with an invalidity, end with an invalidity (except by sheer chance every once in a while, like a blind and anosmic squirrel who finds the occasional acorn, a result may be “true”) or to put in more mundane terms crap in-crap out.
7. And so what does this all mean? I’ll let Wilson have the second to last word: “So what does a test measure in our world? It measures what the person with the power to pay for the test says it measures. And the person who sets the test will name the test what the person who pays for the test wants the test to be named.”
In other words it attempts to measure “’something’ and we can specify some of the ‘errors’ in that ‘something’ but still don’t know [precisely] what the ‘something’ is.” The whole process harms many students as the social rewards for some are not available to others who “don’t make the grade (sic)” Should American public education have the function of sorting and separating students so that some may receive greater benefits than others, especially considering that the sorting and separating devices, educational standards and standardized testing, are so flawed not only in concept but in execution?
My answer is NO!!!!!
One final note with Wilson channeling Foucault and his concept of subjectivization:
“So the mark [grade/test score] becomes part of the story about yourself and with sufficient repetitions becomes true: true because those who know, those in authority, say it is true; true because the society in which you live legitimates this authority; true because your cultural habitus makes it difficult for you to perceive, conceive and integrate those aspects of your experience that contradict the story; true because in acting out your story, which now includes the mark and its meaning, the social truth that created it is confirmed; true because if your mark is high you are consistently rewarded, so that your voice becomes a voice of authority in the power-knowledge discourses that reproduce the structure that helped to produce you; true because if your mark is low your voice becomes muted and confirms your lower position in the social hierarchy; true finally because that success or failure confirms that mark that implicitly predicted the now self-evident consequences. And so the circle is complete.”
In other words students “internalize” what those “marks” (grades/test scores) mean, and since the vast majority of the students have not developed the mental skills to counteract what the “authorities” say, they accept as “natural and normal” that “story/description” of them. Although paradoxical in a sense, the “I’m an “A” student” is almost as harmful as “I’m an ‘F’ student” in hindering students becoming independent, critical and free thinkers. And having independent, critical and free thinkers is a threat to the current socio-economic structure of society.
LikeLike
Duane,
Thanks for continuing to post these ideas.
I just took a few minutes to read through them again. I guess summer is when I have a little more time to think. (Ha, ha, ha….. isn’t that ironic since school is a place where we all should be thinking, though who has the time these days, with SLOs, APPRs, BLAH BLAH BLAH… But, of course, maybe that’s just what the people in charge want… less real thinking.)
I ran into a retired teacher I worked with yesterday. We were at the hardware store in Liberty. He’d heard about the crazy amount of testing we have these days in school. And, like I typically mention when I get into this subject, I said I don’t think most people outside schools really CARE what is actually on these tests. They are just looking for a number to pin on us and our students. Like pin the tail on the donkey at a children’s party. This former colleague of mine got the idea immediately.
It sounds like the piece you like to reference focuses not just on WHAT is being tested but also on HOW that knowledge is acquired. And, of course, many of the so-called “reformers” don’t care about that idea either. I’m sure NY Governor Cuomo, for example, could care less. “Epistemology” for him? Well, that’s for “eggheads” -to use the term from the 1950s. Amazingly, these are the same alleged leaders who then turn around and complain that our country is not as smart as other nations. Their willful ignorance and mocking of philosophy, though, is really the problem.
My son has his learners permit so he drove us all the way to the store yesterday. He’s gearing up to take the road test so I just went back and looked at the ideas you posted in light of his soon-to-be date with a DMV road test examiner. It was fun, Duane. It’s a test that most of us have to take, after all.
I think most people would agree that it’s probably a good idea to have some sort of test to make sure that potential license holders know how to drive a car in some basic sense. But we know that any road test will be only a rough gauge of how good a potential driver will be and we accept that fact. (For example, the “frame of reference” in a typical road test is hardly scientific….Wilson’s “General” frame” right?) I mean, I took a road test but I don’t remember getting any standardized test score for that.
And, we all know that the road test examiner does hold a significant amount of power…. just cruise around, maybe? Or, how about some parallel parking on a busy street? Was the examiner in a bad mood? Was there a lot of traffic that day? We know there are lots of variables that don’t necessarily have to do with our ability to drive. The examiner is a “judge” and we accept that fact, too.
Of course, if we really want to help and protect our kids then wouldn’t we be looking at the types of education and testing that new drivers’ get? I mean, what could be more dangerous than piloting a giant block of metal at high velocity around LOTS of other citizens? Yet, I get the sense that many kids nowadays get even FEWER driver ed hours in public schools than I got decades ago. (Cut from budgets?)
But really, what do all those drivers ed hours and the road test really mean, anyway?
I don’t believe I was ready to drive after I passed my road test back in the 1970s. But that fact didn’t stop me from setting out one night on a four hour trip to NYC after getting out of work at 10 p.m. And, I had just bought a car with a standard shift that I really couldn’t fully work yet. WHAT was I thinking? There was no test that would gauge THAT sort of decision-making in my much younger, quicker brain. But I certainly did learn a lot that particular night going to New York. (That’s the real “story about myself”, not the passing grade I got on the road test.)
No, our culture doesn’t really care about the tests and it doesn’t really care about our kids. I mean we parents and teachers do. But, hell, this political mess called the Common Core is a disaster in the making for our children and our nation.
If only the crap our kids will have to navigate through in our schools these days was as easy as driving down the road.
LikeLike
P.S. Yup…we’re “gagged” AND wearing a blindfold….
LikeLike
Thanks for the response John!
To get a drivers license in Missouri (is that Liberty, MO to which you refer?) one has to take a multiple guess test (General Frame) to get a permit, then drive for X number of hours as verified by a parent/guardian-my ex lied about the number of hours my oldest son drove, ha ha- and then take the driving portion of the test (a Judge frame).
Anecdotally speaking I and my youngest son moved out here to the countryside down a couple of miles from the nearest paved roads. The custom around here is for young kids (as young as 10) to take the wheel and drive on the gravel with mom/dad. When my son got behind the wheel the first time at age 14 he was driving when a county sheriff came around the curve towards us and signaled for him to stop. We rolled down the window and chatted “Boy’s learning how to drive, eh?” “Yessir, first time behind the wheel.” He looked at my son (who was quite tall then 6’2″ or so) and said “Well be careful and drive safely” and went on his way. Needless to say my son was a little taken aback as he thought he was getting caught driving without a permit.
I’ve contended for many years that the main tool in our bag to defeat the educational malpractices of educational standards and standardized testing regimes is the COMPLETE INVALIDITY of said processes. Unfortunately we’ve all been brought up to believe that there is “scientific” validity (at least that is what the psychometricians and edudeformers would like all to believe) to those malpractices. Epistemological and ontological validity form the very basis of any and all “truth” statements. No scientist worthy of that label would lend credence to those concepts unless they have not critically examined them. The “numerization” of society can have serious side effects!!!
LikeLike
Liberty, New York, Duane.
I love the story of you driving with your 14-year-old son.
Thee are plenty of back roads up here, too.
LikeLike
In other words, SOSDD for the testing industrial complex. Profits (for adults) before people (students).
LikeLike
Not sure about gag orders, but I know one thing for certain: I have to suppress the gag reflex in order to read the NY Times these days.
LikeLike
I gag every time I read about these absurdities.
LikeLike
Would pharmaceutical companies, legislators, governors and the medical profession place gag-orders on physicians? Mandate that they are forbidden to report errors, side effects, and potential harm to patients?
It is OUTRAGEOUS placing gag-orders on teachers!
Parents should rise up!
Their children are no longer safe because we are not allowed to report…anything professionally unethical, unprofessional and causing harm.
These are the worst of times…and getting worse.
We must fight this with all our might!
LikeLike
Makes sense. Teachers can’t read or discuss the tools that render critical data that affects everything they are doing.
Just as Arne, Bill, Eli, John, Michelle, Campbell, Moe, Larry and Curly don’t read or discuss anything of true value in education, they just repeatedly implement and promote garbage regardless of results and indicators.
LikeLike
Has NYSED released 3rd grade ELA test questions? Where can I see what they released?
LikeLike
Please explain how, for generations, the New York State Regents Exams were publically released in their entirety within months of their administration. I took at least a dozen of them and they were all challenging. We studied old tests and our class notes and books. It was rare to find an exact question repeated verbatim: can’t recall it happening but maybe it did. Still, the kids who did well in class and worked hard passed. The kids who struggled in class had difficulty passing the test. It seemd logical. So now tests are classsified secrets, teachers forbidden from even reading them? What utter hogwash. And teachers will be evaluated on these results? It’s so twisted.
LikeLike
Today I read an article in the New York Times that stated that releasing the regents exams led to test score inflation. As you say, “utter hogwash.”
LikeLike
My mother used to practice with old regents exams back in the day and she’s 89. Now all of a sudden looking at old tests results in inflated scores?
Anybody who takes the time to “study” for a final shows grit and determination – qualities which are supposedly necessary for career and college ready students. Now studying the material you are to be tested on is inappropriate?
Who’s making up these rules? Godzilla?
LikeLike
Gateszilla
LikeLike
I guess that the test makers would prefer teachers to be like Sargent Schulz in the Hogan’s Heroes television sitcom from the late 60s:
“I see nothing Colonel Hogan.”
LikeLike
When is a gag, not a gag? When it’s a gag!
What nonsense!
LikeLike
Security prevents scrutiny.
LikeLike