The new website testingtalk.org, includes a fascinating discussion of a question on the test. Open the link to tread it.
Here was a comment by another teacher:
“Today my 24 gems sat down to take the NYS ELA test, book 1, day 1. This test consisted of 5 passages and 30 multiple choice questions. I felt the passages were okay readability, but the questions were unlike anything I’ve ever experienced in my 8 years of teaching. Many of the questions were wordy, required my 8/9 years old to flip multiple times referring to paragraph numbers, and included very close answer choices. There were questions neither myself or my colleagues were able to answer with a reasonable degree of certainty. There was no way to “prepare” my students for these type of questions. I’ve been teaching solid reading skills about characters-that characters are more than one way, about plot/setting- the events can impact the story, and how to defend or support your claim or thinking with evidence from the text. We’ve read and practiced non-fiction reading (locating important lines, adding up details to find the main idea, and looking out for an author’s point of view). Every single one of my students have shown growth this year and are avid readers. Unfortunately, this test did not measure CCSS (character development, feelings, locating details, etc) and seemed more like a trick, than a measure to check understanding of passages. My students took almost the entire time to complete their assessment and 2 did not finish. This “test” was disheartening and a complete joke. April fools maybe?”
Disgusted
Author: Mary, Teacher | State: NY | Test: | Date: April 1 at 10:16 pm ET
I am truly disgusted at what these tests have become. As a fifth grade teacher, many of my best readers were in tears before I even handed out the booklets. They were so stressed because of all the pressure. After calming them down and assuring them that they were prepared and should just try their best, I opened up the test booklet and wanted to cry myself. The passages were very long and the questions were absurd. What is the point of analyzing every single sentence in a passage? It’s ridiculous! My students worked hard and used every strategy they knew to help them get through it and I’m really proud of them. However, the looks of frustration and misery on their faces was heartbreaking. The test is too long and they are not given enough time. Why is a reading test timed in the first place? Now they need to endure this for another two days?? Something needs to be done! I’ve seriously lost all respect for this system.
Comments
Author Comment
Anonymous, Teacher
April 1 at 11:56 pm ET Same thoughts in 3rd grade. The answer choices were so close and there were clearly 2 answers that could be reasonable. So disheartening for my readers that are growing everyday…
An 11th grade teacher who gave the Smarter Balanced test wrote:
“I teach in an urban school in Los Angeles. Most students did not finish the ELA portion of the test; the passages were extremely long and the vocabulary was above their reading level. I believe it will take generations for students in American schools to be prepared for a test of this caliber, and a real investment in improving urban schools and communities.”
Diane Ravitch's blog 
These tests are obscene. Demand that the questions be released and subjected to scrutiny!
Don’t allow the creators of the fiendish Common Core College and Career Ready Assessment Program (C.C.C.C.R.A.P.) to hide in the dark.
Testing of public school students is not supposed to take place in some sort of STAR Chamber.
As that great student of the Rheeformish tongue, philologist Don Duane Swacker, Hidalgo, has pointed out, P.A.R.C.C. is spelled BACKWARD in keeping with the practice of the notorious grimoires of the past.
Bob, Your word choice of is so on target:
Obscene….Fiendish……Dark…..Chamber…..Grimoires……
Pearson’s Medieval Torture for children is coming from the minds of emotionally disturbed people. I’ve been using the words “Sinister” “Nefarious” and “Dystopic” a lot too.
Any legitimate educator can recognize these tests and CCSS materials are designed to intentionally confuse, frustrate, and intimidate children; therefore, causing them to suffer from chronic fear and stress. We know that chronic stress from fear and intimidation in childhood can lead to permanent psychological damage. (usually mental illness via personality disorders).
In recognizing this, we can say with all sincerity, “These perpetrators are intentionally hurting children.” This deviant behavior is classic of a bully, a tyrant, a dictator (sociopathic behaviors).
Lloyd Lofthouse had an insightful comment on Diane’s blog yesterday about the business trend of corporations to seek out leaders who have sociopathic behaviors. He posted this article:
http;//www.theguardian.com/commentisfree/2011/nov/07/one-per-cent-wealth-destroyers
I have a new acronym: BCCCCS
The Billionaire Common Core Country Club of Sociopaths are very sophisticated. Instead of sending death squads to us, they have indoctrinated us to drop our children off into their “chamber” every school day to be tortured with Pearson’s brainwashing methods by teachers who have been indoctrinated into this systemic abuse.
( This dysfunction is a sign that we are becoming a master/slave society).
The BCCCCS who are targeting children with this hostile takeover of public education are using Pearson to design the methods of torture. Children are just a “product” to them. They are “breaking children” in order to break up the public school system. (I have a visual image of Abu Ghraib).
This black pedagogy ( Schwarze Padagogik – breaking the will of a child) demonstrates the darker arts of psychology at work by these corporate and governmental sociopaths.
The BCCCCS are “unconsciously” creating a corporate model of sociopathic society while we are all looking on as helpless bystanders. This social emotional regression is leading to Medieval barbarianism, totalitarianism, a dystopian society.
What happens when we say “The Emperor is wearing no clothes” and no one listens because our leaders are all brainwashed or in denial…..should we remind everyone to think of the sociopathic corporate trend that is becoming typical in the US…..
Enron, Wall Street, Bernie Madoff, Lance Armstrong, …fill in as many blanks as you can…….
My recommendation would be for parents to organize like a union and put yellow crime scene tape around every public school in America on test days until we can stop this monster that is taking our children.
Bob, how CAN we demand that the questions be released and subjected to scrutiny? I would love to be part of this effort, but wouldn’t know where to begin. Does anyone have advice?
I would start at your building level.
You will be told no.
Work your way up form there.
Just keep requesting to see your child’s test, the key and the cut scores.
Invite other parents to join you!
This is a SUPERB question. Would love to hear what others have to say.
Organize parents in your area. Meet. Plan.
Come up with some slogans.
Create a logo.
Keep your message simple. One message. A couple slogans. Something chantable.
Get lists of media personnel in your area.
Inundate them with letters.
Prepare some boilerplate letters.
Do petitions on Change.org.
Use Meetup (a SUPERB TOOL) to organize groups of parents to meet and plan.
Create a blog, a website, and a Facebook page.
Leaflet community events.
Attend school board meetings and speak out.
Organize a march on your state capitol.
Write a song.
Do some Youtube videos.
Do a letter-writing campaign.
Make some buttons.
Use PR Newswire to send out a news release regarding an event. Contact all local media before the event to ensure that they will be there to cover it.
Enlist faculty from local universities to assist. Approach them with particular requests.
When you hold events, do something at them to create some drama. Publicize that there will be drama beforehand. You want TV cameras and eyeballs.
“Demand that the questions be released and subjected to scrutiny!”
I have been chanting this since the beginning of mandatory, corporate, standardized testing.
I really don’t see how so many people just accept that these super secret, never seen the light of day, floating cut score tests measure anything at all, let alone are appropriate, unbiased, and fair.
Demand to review any test your child is required to take.
Heck, you are paying for it.
One poster referred to the tests as, “sadistic”
Heck of a job Davie
So much of what I’ve been reading, and especially the quotes in this e-mail, confirm the idea that one goal of this testing is actually to make our students fail. That is completely unacceptable and must be stopped.
The problem, of course, is that in order to generate BIG DATA, these people need for much of their testing to take the form of items that can be answered by bubbling in bubbles. The preferred bubbleable format, of course, is the multiple-choice question.
At the same time, the dummies designing the formats for these new tests flatter themselves that they, for the first time, are testing “rigorous” thinking.
(It would be darkly amusing that people who think as sloppily as these people think should be talking about “rigor” if what they are doing weren’t so terribly, terribly damaging to kids.)
The fact is that the creators of the CCCCRAP tests haven’t thought at all rigorously about what they are doing. Their two goals—bubbleable multiple-choice questions and rigor–are incompatible.
If the questions are to be rigorous, they think, then the student must not be able simply to guess. So, the distracters (the incorrect answers; alternate spelling: distractors) have to be plausible but incorrect.
That’s their goal. Plausible but incorrect distractors.
The problem with that, of course, is that plausible means, well, PLAUSIBLE!!!!
Look up the word PLAUSIBLE. IT means “seemingly or apparently correct.”
So, the incorrect answers are PURPOSEFULLY supposed to be “seemingly or apparently correct” answers!!!!!
I wish I were making this stuff up!!!
Here’s the problem in a nutshell: Multiple choice is generally lousy way to test anything beyond factual recall, but the geniuses designing these tests haven’t figured that out–something most classroom teachers figured out ages and ages ago.
So much for the new, rigorous, higher, revolutionary test designs!!!!!
Several hundred million dollars of taxpayer money have been spent to create this C.C.C.C.R.A.P., which almost no parents nor teachers wanted in the first place!!!!
Plausible incorrect multiple-choice answers.
Ah, what will these “testing experts” come up with next?
Perhaps we can have new ultra rigorous tests in which all the multiple-choice answers are correct but one is a little more correct than the others are! That’s almost what they are producing now–questions for which one of the answers is a little more equal than the others.
Perhaps some people, like the animals of Animal Farm, are more equal than others. Perhaps they are born to plutocrats and go to expensive private schools where they would never be subjected to such abuse.
In short, in order to design questions of these types WELL, you have design them in such a way that they will, inevitably, be difficult. Catch-22. Interestingly, the makers of these standardized tests are becoming victims of their own increased sosphistication in test-making. The problem that they are encountering here should be making them think twice about using such formats AT ALL. But the folks entrusted to envision these tests seem to be slow learners, there. It’s probably going to take a national revolt against their stupid tests for them to start thinking that, duh, they may have made a few pretty serious mistakes in conceptualizing them.
And, of course, they should be rethinking doing standardized testing at all.
I don’t think that there should be anything Common about the kids were turn our in our schoolsm except their love for one another. I want us to develop unCommon kids.
I don’t want to give standardized tests. I want to see people perform in ways that are surprising, innovative, and nonstandard.
We’re not producing widgets here, folks, and schools are not shirt factories circa 1926.
“. . . for them to start thinking that, duh, they may have made a few pretty serious mistakes in conceptualizing them.”
Those “serious mistakes in conceptualizing them” are those epistemological and ontological errors that Wilson has identified that render the educational standards and standardized testing processes completely invalid. No getting around that fact.
They also coat themselves in PLAUSIBLE deniability so that when the sh!t hits the fan the can deflect and deny any culpability.
The more plausible you make your incorrect answers, of course, the more kids will fail the tests, and the more proof the tests will generate that we need to privatize all the schools, fire all the teachers, and replace those teachers with software.
In other words, the more educational deform you introduce, the more you need educational deform.
Of course, it’s possible to write plausible but incorrect answers, but it’s fiendishly difficult to do that for question after question, test after test, and do it well. And, of course, we used to have a name for such questions. We called them “trick questions.”
Call these, if you will, the trick tests. Trick, certainly no treat. Perfect for the Common Core Era, for the day of Son of NCLB, NCLB Undead, NCLB Fright Night II: The Nightmare Is Nationalized.
Mark my word: When these tests are rolled out nationwide, there is going to be an educational policy supernova.
Or, to go back to the previous metaphor, that’s when the villagers, nationwide, will grab their pitchforks and shovels and track the ed deform monsters to their lairs.
But what is this site? What is its bent? ….lotta white people on its “committee” is all I can say: http://testingtalk.org/committee/ Not that this is inherently invalidating, but one does have to wonder about the imperialist nature of education-evaluation (for the record I too am white and am a “parent”). It looks to me like one of Garrison Keillor’s Professional associations of allied organizations ….. not sure my point but sometimes the objections feel like being in an echo chamber.
LOL
(moreover, feedback sent to the site at the address soliciting same (websiteissues@testingtalk.org) bounces)…
http://www.hslda.org/docs/news/2014/201404010.asp?elq=74cff88d93dd45efb7407104812e93f5&elqCampaignId=444
Meanwhile, back at the OECD… http://www.telegraph.co.uk/education/educationnews/10735012/OECD-problem-solving-test-how-good-are-you.html
Plain and simple.. assessments are supposed to be learning tools for both teacher and student. If an assessment does neither… just why is it given? If a test does neither and additionally takes up a HUGE portion of the school year and underhandedly dictates what and how a teacher “instructs”.. this makes its roll out EVEN MORE QUESTIONABLE. This is the golden “April Fools” Day joke. My state started field testing on April Fool’s Day and apparently some states like NY actually tested on April Fools Day. It is over-the-top insulting to force these extremely expensive and harmful tests on a nation… so fitting students would be taking them on the official “day of the prank”… just wish it were a bad prank that would GO AWAY!
Here would be a great prank… let’s make sure all students have a good breakfast before taking this test (perhaps the title one school breakfasts which are loaded with microwaved food in plastic and processed stuff). Then students could take syrup of ipecac and vomit right on the tests. Of course gloved teachers would have to bag the tests (I am sure it is in the manual on how to pack compromised tests like this). Imagine sending them back to Pearson! Now that would have been quite an April Fool’s Day testing statement!!!
Today my 5th grade class was subjected to day 1 of a 3 day Common Core exam that according to New York Governor Cuomo will result in “unfair test results.” He even said students must be protected from these results. The legislature took the extraordinary (not really) step of including a bill in their budget proposal that says districts should not use the results of these tests, unless of course they want to.
Confused yet? During last’s night debate ( not really) on the New York State Assembly floor, Assemblyman Graf asked if it was true that they were voting to reduce a 90 minute test to 90 minutes! ( Yes!, Really, 90 minutes!)
Today, Cuomo announced that he needed to look at the impact of Common Core testing on teacher evaluations. ( Yes, he really said that!) Cuomo said Common Core testing has been rolled out too quickly and it is not a fair indicator of student success.
Has the campaign begun??
As Cuomo fiddles, the students in my class are being put through the fire. They sat today trying to decipher ridiculous questions that asked, among other things, relationships between paragraphs, which paragraphs contribute to the structure of a story, and what should have been included in the summary of a story they read.
The read 6 essays, and answered 42 very difficult and abstract questions. The stories had characters with names that would be unfamiliar to 98% of New Yorkers. They were uninteresting and the students told me later just how hard they struggled to stay motivated. Observing, I witnessed students struggle through the first couple then they seem to just skim and randomly select bubbles to fill in. One student said, “This was the worst 90 minutes of the entire year.”
Students wondered if they failed, students were drained and wondered what they would face tomorrow.
My question to the governor,
Why did we abuse New York’s children today?
http://rlratto.wordpress.com
Thank you for calling it what it is:
Why did we ABUSE New York’s children today?
He’s a phony and it’s political posturing but at least he’s addressing it.
The other CC governors seem to have “relinquished” any responsibility for it. They farmed it out to the private contractor and moved on.
TRAPS not tests, Designed to TRICK, CONFUSE, FRUSTRATE, RATTLE, TIRE OUT, and WEAR DOWN young test takers into FAILING.
Parents,
Please do not allow the bogus test scores produced by Pearson/PARCC to dismantle public education, humiliate, intimidate, and fire teachers, and most importantly, do not allow your sons and daughters to be intellectually defined by this nonsense.
BOYCOTT the tests. CALL your building principal and superintendent to COMPLAIN.
Fight back with all you have. These tests are the spear point of an invasion on public education. Be fierce and fearless.
Excellent!
The kids aren’t writing long enough. I wonder if it’s because they feel they don’t have enough time, or because the format where one has to put text into a small box and the writer can’t see the whole essay while flipping back and forth to the passages is poorly designed.
If you’ve taken a high stakes test that involves lots of writing, as I’m sure many adults and older teenagers have, the most difficult part is managing your time – how long to allocate to each question. Test prep programs for the bar exam do a lot of prep on just that, dividing the time for each essay into blocks yourself and sticking to a schedule.
That must be an absolute bear for younger kids.
“ELA test length and PCR Response
PARCC has received numerous reports of students completing the ELA sessions in times that indicate many students are not writing extended, well-developed responses to the Prose Constructed Response (PCR) prompts. The PCR prompts are intended to elicit extended written responses, requiring students to draw on evidence from the passages they have read to support the points they make in their written responses.”
http://www.parcconline.org/live-field-test-updates-march-26
And here we see evidence of how standardized tests lack merit. Before I make my larger point, I’d note that the biggest flaw of these tests is that students care about their results. As a high school teacher, I can tell you that kids only care about the ACT. State tests are meaningless. If they don’t care, then the results are invalid.
Second, I’ve seen sample SBAC questions and the problems noted above were true. When I read the answer choices, I believed that I could often make a justifiable case for 2-3 answers being correct if I were able to explain my reasoning. Kids would often simple guess from more than one reasonably correct choice. So: if this were designed to allow kids to demonstrate their “critical thinking skills”, then wouldn’t kids be allowed to explain their choice? Not in this instance.
The problem is not new with these tests. Over the years, I have reviewed hundreds of state ELA tests. Often, the problem of the incorrect answers being too plausible was compounded by the stem and distractors being so badly written that a) the correct answer was unintentionally incorrect, b) one or more of the incorrect answers was untintentionally correct, or c) unintentionally, none of the answers was correct.
Those teachers who have shared their experiences need to be careful with their openness, honesty and caring as well as with whom they share their experiences and frustrations. It suggests that they have seen and read the test and have realized how inappropriate it may be for whatever reasons, but they may not have had the permission from New York to have done so. I hope that this blog provides that safe outlet for them.
When one of the fellows I taught with complained about some aspects of the science portion of the CAPT in Connecticut, he went to the Department of Education in Hartford to voice his concerns. He was essentially told that he had not been allowed to read the science portion of the test and to stop complaining or he could potentially lose his certification to teach in the state.
So much for open and honest dialogue and a receptive response to constructive criticism!
Reading these statements makes me want to cry. I have retired but I am still in tune with the pulse of pain and the downturn of morale caused by this testing mania.
I recall the faces of some students who worked so hard to achieve a good score on their tests. The pressure of VAM puts such a cloud of despair over the teaching and achieving since it is hardly apparent as to how test questions will be posed.
This endeavor is so misguided.
I am impressed that a teacher could actually read the test. In my state you sign an oath you will not read any test material, and if you are responsible for reading any portion of the test to students, you are sworn to secrecy about any content. We sign oaths before we test, and after we test, and have proctors that watch us like hawks to make sure we don’t cheat. Based on the release tests they have taken in the past, the test usually has 4 passages including a paired passage plus a field test passage and questions that do not count against the student’s score — of course, it does add to the work load and stress factor. In my grade, students must make a passing score to be promoted to the next grade and the questions are ridiculously convoluted to challenge their “higher level thinking skills.” (Not to worry, some genius came up with some games to help students eliminate “obviously” wrong answers from their choices so they only have to struggle between two final choices. There is class time well spent — playing games to help students “master” the test.)
My state does not follow Common Core. We have been testing for decades but TEKS are now aligned to CC and it would seem so are the our tests – big advantage for Pearson. Sadly, because these tests have been a part of our educational system for years, they are an accepted and expected part of the school year. For those that work in education, we know these new tests are nothing like those in the past which were pretty pointless even then. Even as the nature of the test has become more and more structured and prohibitive, they are so standard there is very little push back. We have been conditioned to think all this is normal but there is nothing productive, informative, or “normal” about any of it.
So, I am off to a second day of testing, where my room is cloaked with paper to cover any information useful to test takers (including the calendar), my student desks pushed apart with privacy screens to block wandering eyes, and my perch is at the front of the room so I have a place to watch when I am not “actively monitoring.” No child can move without permission and only one child can leave the room at a time. I can leave when another certified teacher takes my place and there is a document to sign in and sign out to show when I left, returned, and who monitored my class. We will eat lunch in the room to be sure they don’t discuss the test. Tests are precisely timed for four hours, but it is a very long day — and for those unfortunate students that do not successfully run this gauntlet, they will get to do it again — and again, until a committee passes them on to the next grade. No child will be left behind.
Bob, The word choices in your first comment is so on target:
Obscene….Fiendish……Dark…..Chamber…..Grimoires……
Pearson’s Medieval Torture for children is coming from the minds of emotionally disturbed people. I’ve been using the words “Sinister” “Nefarious” and “Dystopic” a lot too.
Any legitimate educator can recognize these tests and CCSS materials are designed to intentionally confuse, frustrate, and intimidate children; therefore, causing them to suffer from chronic fear and stress. We know that chronic stress from fear and intimidation in childhood can lead to permanent psychological damage. (usually mental illness via personality disorders).
In recognizing this, we can say with all sincerity, “These perpetrators are intentionally hurting children.” This deviant behavior is classic of a bully, a tyrant, a dictator (sociopathic behaviors).
Lloyd Lofthouse had an insightful comment on Diane’s blog yesterday about the business trend of corporations to seek out leaders who have sociopathic behaviors. He posted this article:
http;//www.theguardian.com/commentisfree/2011/nov/07/one-per-cent-wealth-destroyers
I have a new acronym: BCCCCS
The Billionaire Common Core Country Club of Sociopaths are very sophisticated. Instead of sending death squads to us, they have indoctrinated us to drop our children off into their “chamber” every school day to be tortured with Pearson’s brainwashing methods by teachers who have been indoctrinated into this systemic abuse.
( This dysfunction is a sign that we are becoming a master/slave society).
The BCCCCS who are targeting children with this hostile takeover of public education are using Pearson to design the methods of torture. Children are just a “product” to them. They are “breaking children” in order to break up the public school system. (I have a visual image of Abu Ghraib).
This black pedagogy ( Schwarze Padagogik – breaking the will of a child) demonstrates the darker arts of psychology at work by these corporate and governmental sociopaths.
The BCCCCS are “unconsciously” creating a corporate model of sociopathic society while we are all looking on as helpless bystanders. This social emotional regression is leading to Medieval barbarianism, totalitarianism, a dystopian society.
What happens when we say “The Emperor is wearing no clothes” and no one listens because our leaders are all brainwashed or in denial and part of the dysfunctional system……..should we remind everyone to think of the sociopathic corporate trend of greed and corruption that is becoming typical in the US…..These are a few I Googled:
Enron, Wall Street, Bernie Madoff, HealthSouth, Allen Stanford, Tyco Ltd, Arthur Andersen, Bearn Stearns Inc, Adelphia, Global Crossing Ltd, Martha Stewart, Lance Armstrong, WorldCom,…and can we include all the corporate and gov “spying” from Hewlett-Packard Spying Scandal to NSA Domestic Spying Program.…fill in as many blanks as you can to get the picture…..and I think there was something about Bill Gates stealing from Steve Jobs?
My recommendation would be for parents and teachers to organize like a union and put yellow crime scene tape around every public school in America on test days until we can stop this monster that is taking our children.
Re: Corporate Sociopathy: Here is one of the links from Lloyd Lofthouse April 1:
http://www.theguardian.com/commentisfree/2011/nov/07/one-per-cent-wealth-destroyers
Lloyd,could you please post that comment again to include all the links:
oops….grammatical correction…..Bob, The word choices in your first comment ARE so on target….and they are and are and are!!!
All the associated problems described above pale in comparison to the epistemological and ontological invalidities that the educational malpractices educational standards and standardized testing. With no logical foundation how these malpractices continue (other than the current paradigm is so strong that most can’t resist it’s siren song) is beyond me.
“Facts are less dangerous than theory; despite the promise of the Enlightenment, most people use up far more energy defending their mythologies [educational standards and standardized testing] than in searching for facts; the world is full of answers looking for questions, and significant questions are rather an endangered species.” (not sure of author, if you know let me know)
Folks, we need to attack the beast at its heart which is its complete invalidity as proven by Noel Wilson in “Educational Standards and the Problem of Error” found at:
http://epaa.asu.edu/ojs/article/view/577/700
Brief outline of Wilson’s “Educational Standards and the Problem of Error” and some comments of mine. (updated 6/24/13 per Wilson email)
1. A quality cannot be quantified. Quantity is a sub-category of quality. It is illogical to judge/assess a whole category by only a part (sub-category) of the whole. The assessment is, by definition, lacking in the sense that “assessments are always of multidimensional qualities. To quantify them as one dimensional quantities (numbers or grades) is to perpetuate a fundamental logical error” (per Wilson). The teaching and learning process falls in the logical realm of aesthetics/qualities of human interactions. In attempting to quantify educational standards and standardized testing we are lacking much information about said interactions.
2. A major epistemological mistake is that we attach, with great importance, the “score” of the student, not only onto the student but also, by extension, the teacher, school and district. Any description of a testing event is only a description of an interaction, that of the student and the testing device at a given time and place. The only correct logical thing that we can attempt to do is to describe that interaction (how accurately or not is a whole other story). That description cannot, by logical thought, be “assigned/attached” to the student as it cannot be a description of the student but the interaction. And this error is probably one of the most egregious “errors” that occur with standardized testing (and even the “grading” of students by a teacher).
3. Wilson identifies four “frames of reference” each with distinct assumptions (epistemological basis) about the assessment process from which the “assessor” views the interactions of the teaching and learning process: the Judge (think college professor who “knows” the students capabilities and grades them accordingly), the General Frame-think standardized testing that claims to have a “scientific” basis, the Specific Frame-think of learning by objective like computer based learning, getting a correct answer before moving on to the next screen, and the Responsive Frame-think of an apprenticeship in a trade or a medical residency program where the learner interacts with the “teacher” with constant feedback. Each category has its own sources of error and more error in the process is caused when the assessor confuses and conflates the categories.
4. Wilson elucidates the notion of “error”: “Error is predicated on a notion of perfection; to allocate error is to imply what is without error; to know error it is necessary to determine what is true. And what is true is determined by what we define as true, theoretically by the assumptions of our epistemology, practically by the events and non-events, the discourses and silences, the world of surfaces and their interactions and interpretations; in short, the practices that permeate the field. . . Error is the uncertainty dimension of the statement; error is the band within which chaos reigns, in which anything can happen. Error comprises all of those eventful circumstances which make the assessment statement less than perfectly precise, the measure less than perfectly accurate, the rank order less than perfectly stable, the standard and its measurement less than absolute, and the communication of its truth less than impeccable.”
In other word all the logical errors involved in the process render any conclusions invalid.
5. The test makers/psychometricians, through all sorts of mathematical machinations attempt to “prove” that these tests (based on standards) are valid-errorless or supposedly at least with minimal error [they aren’t]. Wilson turns the concept of validity on its head and focuses on just how invalid the machinations and the test and results are. He is an advocate for the test taker not the test maker. In doing so he identifies thirteen sources of “error”, any one of which renders the test making/giving/disseminating of results invalid. As a basic logical premise is that once something is shown to be invalid it is just that, invalid, and no amount of “fudging” by the psychometricians/test makers can alleviate that invalidity.
6. Having shown the invalidity, and therefore the unreliability, of the whole process Wilson concludes, rightly so, that any result/information gleaned from the process is “vain and illusory”. In other words start with an invalidity, end with an invalidity (except by sheer chance every once in a while, like a blind and anosmic squirrel who finds the occasional acorn, a result may be “true”) or to put in more mundane terms crap in-crap out.
7. And so what does this all mean? I’ll let Wilson have the second to last word: “So what does a test measure in our world? It measures what the person with the power to pay for the test says it measures. And the person who sets the test will name the test what the person who pays for the test wants the test to be named.”
In other words it measures “’something’ and we can specify some of the ‘errors’ in that ‘something’ but still don’t know [precisely] what the ‘something’ is.” The whole process harms many students as the social rewards for some are not available to others who “don’t make the grade (sic)” Should American public education have the function of sorting and separating students so that some may receive greater benefits than others, especially considering that the sorting and separating devices, educational standards and standardized testing, are so flawed not only in concept but in execution?
My answer is NO!!!!!
One final note with Wilson channeling Foucault and his concept of subjectivization:
“So the mark [grade/test score] becomes part of the story about yourself and with sufficient repetitions becomes true: true because those who know, those in authority, say it is true; true because the society in which you live legitimates this authority; true because your cultural habitus makes it difficult for you to perceive, conceive and integrate those aspects of your experience that contradict the story; true because in acting out your story, which now includes the mark and its meaning, the social truth that created it is confirmed; true because if your mark is high you are consistently rewarded, so that your voice becomes a voice of authority in the power-knowledge discourses that reproduce the structure that helped to produce you; true because if your mark is low your voice becomes muted and confirms your lower position in the social hierarchy; true finally because that success or failure confirms that mark that implicitly predicted the now self-evident consequences. And so the circle is complete.”
In other words students “internalize” what those “marks” (grades/test scores) mean, and since the vast majority of the students have not developed the mental skills to counteract what the “authorities” say, they accept as “natural and normal” that “story/description” of them. Although paradoxical in a sense, the “I’m an “A” student” is almost as harmful as “I’m an ‘F’ student” in hindering students becoming independent, critical and free thinkers. And having independent, critical and free thinkers is a threat to the current socio-economic structure of society.
Duane Swacker:
I do not think that the mental health and welfare of children would “pale” to anything you describe.
Everything else is a smoke screen covering the hidden agenda, which is to break the public school system and “break” children in the process.
Children are only “products” to the corporate mind of billionaires. They think children can be “processed” like canned food.
Someone used the word “sociopath”. That is appropriate diagnosis.
I must respectfully ask all teachers to check their grammar before hitting the send button on these post. The number of grammatical mistakes above is far too high and that undermines the messages. There are subject-verb agreement problems, singular-plural agreement problems, usage problems (neither-or rather than neither-nor, etc.) and flat out coherence problems. Don’t let your passion drive you to hit the “send” or “post comment” button before you have taken the time to inspect what you just wrote.
One could say the following about your post, Stephen:
a. The word post in “on these post” should be plural.
b. A comma should appear before the coordinating conjunction in the second sentence because that conjunction separates two main clauses. Some school grammars make this comma optional if the main clauses are short, but many do not, and “short” is vague.
c. If the “There are” in the third sentence is to be locative rather than merely existential, then the location has to be given in the sentence. Otherwise, the expression is ambiguous between the specific and the more general usage: “There are . . . in these posts.”
c. The items in the series in the third sentence should be separated by semicolons because semicolons are used to separate items in a series that themselves contain commas, and, of course, a semicolon should appear before the coordinating conjunction.
d. A hyphen should be used to separate “flat” and “out” in sentence three (“flat-out coherence problems”) and for other such adjective or participle + preposition or particle combinations used as adjectives (e.g., “dressed-up children,” “go-to person,” “clean-up procedures”).
e. “Send” and “Post Comment” should both have initial caps.
One could make such comments, but it would be pedantry to do so. Posts are conversation. They are not finished, edited copy. As every reporter knows, verbatim transcripts of conversation, even among very sophisticated speakers, are full of “errors” in grammar and usage, and blog entries will be, too. That’s inevitable and, I think, excusable. Alas, I’ve often hit the Post Comment button and sent some blog entry with errors in it off into the cloud. We all have. I think that tolerance is in order, that we should judge not lest we be judged. That said, having sent too many inadvertant errors off into the blogosphere, I am trying, these days, to be a bit more careful. However, even very capable writers and editors, clearly, will not give the same level of proofreading attention to these hastily written posts that they give, say, to a submission for publication, and even published works will contain errors. I have never read an error-free book.
While I see your point, I was under the impression that, since many of those who post are educators, we might be expected to use grammar effectively, particularly subject-verb agreement and correct tense applications.
Pardon us if we sometimes let passion, emotion,
and haste get the best of our grammar. And don’t forget, there is no way to edit post facto.
And remeber, teachers who write in glass blogs shouldnt throw stones
It is truly frustrating when I can’t correct a blog post! Sometimes I am in a hurry or am fired up about something. I have made several errors, but most of those have been due to the choices of my android phone.
Yes, we should all attempt to present our ideas as professionally as is possible, but again, I think that on a blog with no editing features, tolerance is in order. I have edited books by some of the most well-known scholars and public intellectuals in the United States and can tell you that everyone (I, included) needs an editor. Harriet Monroe, who edited Poetry magazine for decades, once wrote that none of the big poets she published, with the exception of T.S. Eliot, could spell worth a ______. That’s interesting, given that she published Pound, Stevens, Yeats, William Carlos Williams, and just about every other well-known poet of the era.
Let he or she who is without fault cast the first stone.
“Plausible but incorrect distractors.” …Bozos on the bus…
I don’t need to check my “Grammar”…She’s still in the cemetery…
Thanks for letting us know about this wonderful site, Diane. I’m directing all my friends and colleagues, and letting them know that this is where they will see actual honest feedback….unlike the PARCC website or Twitter feed! Oh my goodness! If you haven’t checked it out, you should. Makes for a good laugh! 🙂
I don’t give the tests but is there a rule, yet, about taking pictures of the questions with your cell phone?
Yes.
Teachers are threatened with removal of license for even looking at the tests.
Some teachers are, on occasion ,asked to read them to ESOL or special ed students. So, some have seen the mess.
This obsessive need for surveillance is obviously paranoia and one more example of some pretty messed up people in charge!
It sounds like the CIA and NAS are now working for Pearson?
In Ohio, testing protocol has always required teachers to focus on student behaviors such as being on tasks, needing a restroom break or pencil sharpened. No talking allowed during breaks. Teachers are not allowed to look at any questions or photograph questions. It was considered cheating and called for firing if discovered.
We have always had access to practice tests and old test questions through the ODE website.
However, questions are changed every year, so coaching isn’t possible.
Testingtalk.org serves a really important function. However, teachers do run considerable risk when, in their posts, they talk about these tests with any specificity.
How UNETHICAL is it for a teacher to give a test that they have no clue as to what is on it?
These tests aren’t about what students “know” in a factual sense. The tests are supposedly designed to see how well the students can handle “cold” information. The test seeks to find out whether the teacher has shown students ways to break information into bits and pieces, how well the students can get into authors’ heads, and to make decisions about that information.
Often the test questions are difficult to interpret. The choices may have nuance that is developmentally inappropriate for some children, if not for most children.
Test questions are difficult to create, even when done by a committee of teachers from the same grade level. Multiple choice answers can be seen in different ways. Essay questions are open to interpretation. Often these are scored by minimum wage part-time workers who may or may not score accurately.
For these reasons as well as reasons mentioned by others, I find this current testing to be inappropriate.
Teachers no longer give their test>
They are no longer teachers>
They are now “monitors”>
This is a transition period>
Soon monitors will be replaced by robots.
That seems to be the plan.
Good question. Maybe it’s their response to:
“We spend too much time teaching to the test…”
Deb, I think you are preaching to the choir along with the rest of us…..Amen!
In Texas TEA gets STAAR scorers off “Craigslist”.
After seeing the dysfunctional test questions that I observed students struggling over
while I was monitoring the 4th grade STAAR, I think Pearson gets their test developers from “Craigslist” also!
The tests were funded by USDE on the assumption that you did not need to tie a test to curriculum, only to standards. This dumb premise was discovered after USDE awarded $300 million for test development of the CCSS–half to the PARCC consortium, half to SMARTER consortium. Upshot: Two months after USDE put $300 million on the table for test development, each consortium asked for and received $15 million for the curriculum work. Never mind that the CCSS were marketed as unrelated to curriculum
I suppose we can be grateful that PARCC’s original and approved proposal to USDE is not yet being piloted in schools. The proposal called for called for NINE assessments every year “rather than on one test at the end, to allow for instructional adjustment and extra support to students who need it.” I constructed a spreadsheet to understand what was to be tested when, and who was scheduled to get the results of the tests and for what intended purposes. The analysis is still in my files.
PARCC’s request for the missing curriculum work was filled with puffery. It included an uncertain “could” and firm “will” approach to producing model courses and ancillary curriculum materials (for their tests) by the end of 2011.
“Each model unit could include components such as: instructional materials; formative activities that would give teachers information they need about student understanding relative to the CCSS and PARCC assessments; professional development materials for educators; and tools to inform conversations between principals and teachers, teachers and students, and teachers and parents about the results of the through course assessments. The units developed by PARCC will serve as powerful models for others to develop similar tools for other standards or grades, and will help states and districts evaluate the quality and alignment of similar tools in the market.” Partnership for Assessment of Readiness for College and Careers. Source: Proposal for supplemental race to the top assessment award. Retrieved from http://www.edweek.org/media/parccsupplementalproposal12-23achievefinal.pdf p. 4).
Current tests, in addition to being conceptually mangled from the get-go, are addressing a subset of all of the 1,620 CCSS (including parts a-e), and these items are being conjured and tested as if all students have had instruction in the various grade-level prerequisites. The tests are based on curriculum materials that have not, I think, been independently vetted as CCSS compliant and unlikely to similar to any that are generally available in many schools. A legitimate testing program for any grade-by-grade “reform” on the huge near-national scale as the CCSS should be a rolled out year-by-year for thirteen years, with a beginning in Kindergarten and all of the rest that might give legitimacy to a grade-level test.
Even if pre-requisite instruction and year-to-year emphases called for by the CCSS made sense (and it does not), the test designers are obliged to offer up some rationale for the distribution of items according to the “anchor” CCSS. They should also provide information on how these groups of items are weighted in the overall score. The rationale for this distribution and weighting should be public, and determined by a consensus process similar to that used for the National Assessment of Educational Progress.
In other words, it is time for transparency and intelligible documentation for this whole investment, in forms suitable for the general public and for review by independent experts in test development. Perhaps a Congressional investigation, complete with aome good timelines of key events; federal dollars spent; additional funds from state agencies, foundations, and for-profit companies and so on.
on the DUMB assumption that you did not need to tie a test to curriculum, only to standards.
AMEN!!!!! YES YES YES YES YES!!!!
Tests should cover curricula, and no national agency should specify curricula. Therefore, no national agency should be creating mandatory tests. QED.
I wonder if NYSED had to file a copy of each test with USDOE to show compliance with the requirements of NCLB? If so, they might be FOIL-able.
One technique that I used to use in my classes was this:
I would say:
“As you know, you are going to have a test on Friday. Now, some of you may think, ‘I wonder if he used the same test last year? Maybe I could find someone who had this class last year and has a copy of those test questions. Maybe that person would sell them to me. If I could do that, I COULD GET A PERFECT SCORE!!! That would be PRETTY GREAT, huh!!!???”
“Except, I hope, most of you would have moral qualms about doing such a thing.
“Well, I am going to save you that trouble and money and any sleep you might lose if you yielded to that temptation.
“You see, I happen to know someone who was in this class last year, and I laid my hands on his copy of the test. It’s right here. And I’m not going to sell it to you. I am going to give it away. You won’t have to look for someone who took this class last year. You don’t have to pay a penny for the questions. And you won’t have to lie at wake at night thinking, ‘I’m a cheater.’
“Here are the questions. The questions you will see on Friday will be IDENTICAL TO THESE.
“Nice, huh?
“So, there’s no reason why you won’t all get perfect scores.”
cx: awake, not at wake, of course