This is a thoughtful and important article by Mark Weber (aka Jersey Jazzman), who teaches in public school in New Jersey and is earning his doctorate in statistics at Rutgers.
He notes that both the New Jersey Star-Ledger and the New York Post were outraged–outraged!–that NJ Governor Phil Murphy plans to abandon the PARCC exam, which is aligned with the Common Core. They accuse Murphy of kowtowing to the lousy teachers’ unions and trying to dumb down the test.
But he points out that PARCC and NJ’s previous standardized test (NJASK) produced the same results.
This is worth your while to read as you will learn a lot about standardized testing and its limitations.
Diane Ravitch's blog 
He has many great points, but I have profound disagreement with his support of the Accountability Movement.
it surely does begin to feel like a pathetic need to endlessly compare rotting apples to rotting oranges: so many are still holding on to the TESTING IS SACROSANCT message when each year it is increasingly evident that the testing game is dangerously — and devastatingly — corrupt
Correct me if I’m wrong, but I get the impression that he was interested in the discrepancy between high and low socioeconomic schools’ test scores not as a way to hold those sluggards’ feet to the fire but as a way to make sure that state resources were directed to where they were most needed.
As I read Weber’s post, I was reminded of my experience with testing over the span of my career. When I first started in New York, there was testing in grades 3 and 8, and Regents tests in the high school for select students. Then, came the Basic Competency Tests in the high school in order to press for higher standards. Then, the state imposed certain Regents tests as a graduation requirement. “All roads lead to the Regents.” As an ESL teacher, I had to administer some required tests that were only for ELLs. With the advent of NCLB, the testing increased exponentially in all grades with ELLs being subjected to double testing. By the time I retired, my students were losing twenty-eight instructional morning due to onerous over testing. I can honestly say all this testing failed to make any instructional program or teacher “better.” All the high stakes testing accomplished was to undermine the confidence of struggling students.
Standardized tests are not great diagnostic tools, and the results are delayed so that that have little instructional value. Local or teacher made diagnostic tests have far greater value informing instruction. Overall, more testing does not improve education. I wish the media could understand this along with the wishy washy term “proficiency,” shown in Weber’s movable line graphic. Proficiency is arbitrary, and “reformers” have used this to their advantage. If they want to send more students to charters, simply move the proficiency mark to cast a wider net. Along with the “bottom 5%” which we know in the bell shaped distribution of standardized test scores, we will always have ready supply of students ripe for privatized picking.
“Local or teacher made diagnostic tests have far greater value informing instruction.”
Someone needs to do a Foulcaldian history of how the teaching profession has come to be seen as a “diagnostic” problem away from a teaching and learning process. When, where, how, why is there some much discourse around the standardized test being a diagnostic tool? When, where, how and why has the teaching profession turned to/into a medical discourse of abnormalities to be diagnosed?
The continuance of that discourse of diagnosable abnormalities in a student’s learning only serves to normalize/make common the idea that diagnostic standardized testing is the answer to America’s supposed “failing” public schools while at the same time that malpractice harms every single student in one fashion or another.
The only reason, yes, THE ONLY REASON, any assessment of a student’s work, abilities, efforts should occur is to help enable to the student to further their own assessing and knowledge of their own learning (metacognition, eh).
“But, but we have to determine disabilities”. Yes, and that can be done on an individual basis by the teacher and the parents/guardians and student without having to put all the other children through the standardized test wringer and/or through any other assessment that does not further the above-stated purpose.
The teaching and learning process has been so bastardized by this medical diagnosing discourse as to become just a series of malpractices foisted upon the students, hindering their learning and not fulfilling the purpose of public education: “The purpose of public education is to promote the welfare of the individual so that each person may savor the right to life, liberty, the pursuit of happiness, and the fruits of their own industry.”
We don’t have to use the term diagnose, although I get it from the reading courses I took to be a certified reading teacher. Determine is an OK term if we are talking about determining where to go with instruction based on student needs.
What I am getting at is that the whole of education has changed in its focus from one of aiding, helping a student and showing the student how to better learn whatever the subject matter to one of determining, or as in medicine, an “abnormality”, an “illness” of the student in relation to what is “normal”, i.e., against a supposed standard. And this not only applies to the few struggling students or to those who may have a learning disability but to all students. What is the normal learning behavior? How is that normal pedagogical behavior determined? Who determines that aspect? It is all tied to the “standard” as defined by someone far away from the individual, you know, as the standards are written-“The students will be able to. . . .”
Now I have no problem with curricular goals, something to stride towards. The problem lies in the thinking that ALL students will progress at the same rate, age, etc. . . and when they don’t they must be diagnosed as abnormal against the standard.
This shift by the teacher away from being a guide to a diagnostician has hindered the teaching and process by having the teacher focus on supposed diagnostic testing and supposed remediation (and as an aside medication for children that may not be necessary). I’m talking of a broad historical trend that should be challenged as a malpractice as we know the standards and testing regime harms many students and is in essence state sponsored discrimination with some rewarded and others castigated. The teaching and learning process should be one in which the individual student can develop however/whatever each one desires in their own learning.
I’m still hashing out this problem, so my explanations are not as cogent as they might be.
‘ “But, but we have to determine disabilities”. Yes, and that can be done on an individual basis by the teacher and the parents/guardians and student without having to put all the other children through the standardized test wringer and/or through any other assessment that does not further the above-stated purpose.’
Which is exactly why the idea of IEPs for everyone horrified me. When we went through a case study process and developed an IEP for a student, it took many, many hours, and every year I spent additional hours updating those IEPs. In fact, when I documented the total number of hours worked, I worked more than 365 days a year in the 185 days for which I was paid. I’m sure there are many teachers who can relate to that. The school district finally agreed to pay special ed teachers for up to three hours of extra time per week. We had to put that extra time in at school and document it, which was generally okay. I always came in early and left late, so I just shifted IEP work that I used to do at home to school and devoted more at home hours to instruction related tasks. The thought of writing and managing IEPs for all students would give teachers nightmares.
And that concept of IEPs for all is really what computerized Competency Based Education is about. Instruction/training (I hesitate to call it teaching and learning) that is supposedly individualized (it’s not) to meet the supposed needs of a student, all without ever having someone personally interact with a student.
Duane,
Glad you jumped in and offered some Wilson voice of reason!!! Please quote more Wilson.
How about this concerning validity and the harms caused by invalidities:
“To the extent that these categorisations are accurate or valid at an individual level, these decisions may be both ethically acceptable to the decision makers, and rationally and emotionally acceptable to the test takers and their advocates. They accept the judgments of their society regarding their mental or emotional capabilities. But to the extent that such categorisations are invalid, they must be deemed unacceptable to all concerned.
Further, to the extent that this invalidity is hidden or denied, they are all involved in a culture of symbolic violence. This is violence related to the meaning of the categorisation event where, firstly, the real source of violation, the state or educational institution that controls the meanings of the categorisations, are disguised, and the authority appears to come from another source, in this case from professional opinion backed by scientific research. If you do not believe this, then consider that no matter how high the status of an educator, his voice is unheard unless he belongs to the relevant institution.
And finally a symbolically violent event is one in which what is manifestly unjust is asserted to be fair and just. In the case of testing, where massive errors and thus miscategorisations are suppressed, scores and categorisations are given with no hint of their large invalidity components. It is significant that in the chapter on Rights and responsibilities of test users, considerable attention is given to the responsibility of the test taker not to cheat. Fair enough. But where is the balancing responsibility of the test user not to cheat, not to pretend that a test event has accuracy vastly exceeding technical or social reality? Indeed where is the indication to the test taker of any inaccuracy at all, except possibly arithmetic additions?”
A Little Less than Valid: An Essay Review
http://edrev.asu.edu/index.php/ER/article/view/1372/43
My major concern is what I have seen PARCC push away in the elementary classroom. The intense „hands-on“ of precise handwriting and cursive instruction has been largely sidelined across the state NJ to prepare students for a computerized test. I would not object to the test if it were administered pencil and paper in the elementary grades with re-emphasis on the hands-on activities – with introduction of computers later (gr. 6) along with thorough keyboarding instruction. I have seen great benefits of traditional manual activities in elementary grades and feel it has been a great loss as they have been squeezed. I am concerned students may be using technology increasingly as a crutch, rather than a tool. I am also concerned that spell check, grammar check technology and the like has come in-between a teacher’s authentic assessment of a child’s weaknesses and strengths early on.
While the digital nature of standardized tests is a concern, it’s a very minor one, like worrying that the car that’s driving down the highway at 100 MPH on fire has a headlight out.
The real problem is the tests themselves for all the many reasons Senor Swacker has pointed out ad nauseum. In fact, this post is crying out for Sr. Swacker’s input – am I the only one who misses him? For the life of me I fail to understand JJ’s continued devotion to the test-based “accountability” regime, if only we could find the right test.
I totally and respectfully disagree. Students need hands on in the early grades. We have produced a generation of hunt and peckers that can hardly print and forget script. Look at the German education in elementary school. Far superior with NO computers and NO Smartboards and a shorter day! Digital nature of the test is the MAJOR concern – but I agree that teacher eval should not be connected with these tests. Look at the beautiful French instruction of script here https://youtu.be/QLF-f1RiFS0 – it is a child’s first “engineering” task and teaches precision.
Perhaps you misunderstood my comment. I’m saying, get rid of the tests. Whether they’re digital or pencil and paper, they are harmful no matter what. I agree with you that digital instruction/testing is harmful. Yes, kids should be learning to write (including in cursive), not type. If we get rid of the tests (including the new and improved “competency” based embedded tests), then schools have more time to teach handwriting and everything else kids should be learning.
Dienne, we agree about that. Duane Swacker is right. The standardized testing obsession should grind to a screeching halt. The results are a measure of family income and education.
Sorry for jumping to conclusions. I am happy we agree. I do know that in order to receive fed aid – at least NJ has to follow some sort of testing.
I wish that one state would defy the federal mandates and say no to standardized testing.
The federal government has never ever withheld aid to an entire state.
Let’s see Betsy DeVos (Ms. Local Control) try it.
Thanks, Dienne and Diane!
But you have to know this one is coming Diane, eh!
“The results are a measure of family income and education.”
No, they are not a “measure” at all in any sense of the word. Is there a correlation between those two factors? Of course there is, and it has been known from almost the beginning of standardized testing. But a correlation is not a measure.
The ingrained, quite subtle usage of seemingly innocuous words can cause great harm. Great harm in this case meaning helping in the continuance of the standards and testing malpractices by lending a sheen of authenticity-well they measure something so it must be accurate and a good thing-that shouldn’t be there.
I concur with the other posters that Weber gives way too much credence to the falsehoods and invalidities that are standardized testing. And yes, that particularly invalid and false malpractice should be stopped immediately and any and all resources dedicated to smaller class sizes-the only guaranteed “reform” that will ever work to truly enhance the teaching and learning process.
Utah tried to push back against the feds, as far as opting out is concerned. Opting out is in Utah law, but the requests from Utah to avoid the 95% testing rate were dismissed by the feds.
So now, parental choice can ONLY happen with charters, NOT with refusing standardized tests.
See here; https://www.sltrib.com/news/education/2018/07/13/students-who-opt-out/
Yes, Duane Swacker, pleae do comment more hère. You always make the most sense when it comes to testing!
I’m grateful to public school teachers for advocating on behalf of public schools.
I was reading about the teachers in Arizona and this stuck out:
“It took us going down there to see what on earth was being perpetrated in the name of schools,” Lewis told The 74. The ESA expansion “really stood out, because not only were they slashing funding, they were going to divert it from public schools to private schools. And that’s what didn’t seem right.”
Ed reformers lock-step promoted the voucher experiment in Arizona, that’s true, but what they neglected to publicize was that ALONG WITH THE VOUCHERS they slashed public school funding by more than 30%.
The ed reform borg ignores public schools so the part where they harmed public schools wasn’t covered amid all the voucher cheerleading, but look what these “education advocates” did to public school families in that state!
They are destroying public schools WHILE expanding vouchers and charters. It’s a one-two punch. Ed reformers in Arizona screwed every public school student in the state while showering funding on private schools. Actively harmful to public school families.
Jersey Jazzman is really good at data collection. I wish he’d look at what happens to PUBLIC schools in states where ed reformers capture state government.
Looking at “choice” isn’t enough. There’s another side to this equation. They promote charters and vouchers, sure, but they also harm public schools DIRECTLY. They’re targeting public schools for eradication and they’re doing it WHILE 90% of students are in public schools.
A whole generation of public school students will be harmed by these “education advocates” and no one pays any attention to them because they attend the unfashionable public school sector.
Chiara, the 90% number is inaccurate unless you think that charters are public schools.
9% of students attend private, independent and religious schools.
6% attend charters.
85% attend public schools.
Thank you.
Scott Stump of Colorado, to be Assistant Secretary for Career, Technical, and Adult Education at the Department of Education.
Mr. Stump is the Chief Operating Officer for Vivayic, Inc., a learning solutions company based in Lincoln, Nebraska.
I suppose we’ll be seeing the public employees at the US Department of Education selling Vivayic product now.
Don’t buy. Find someone who isn’t captured and conflicted and get advice from them.
Ed reformers are sales reps. nothing more. Treat them as sales reps.
Posted at OpEd news.
https://www.opednews.com/Quicklink/What-the-Teacher-Bashing-P-in-Best_Web_OpEds-Diane-Ravitch_Other_Standardized-Testing_Standardized-Tests-180717-131.html#comment706696
My comment has embedded link at the above address.
THE EDUCATIONAL INDUSTRIAL COMPLEX OWNS THE MEDIA” they are not going to tell our people how they are destroying our democracy –even as they demolish the only road to income equality! Schools fail when y the professionals are removed. By replacing competetent professionals with trained civil servants,the charlatans who run the show into the ground, are ending democracy, which depends on shared knowledge!
BTW… The Ravitch blog is a treasure. Everyone should read at least follow her posts.
The posts are to articles by educators across America. It is where I get the latest on the war on Public Education, that began with an assault on teachers, that took out tens of thousands of our most experienced, dedicated professional practitioners..i.e teachers. Also, here at OEN, see my many series which follow the devastation of our public schools, as the INSTITUTION OF PUBLIC EDUCATION is decimated by the ‘reformers’ who sell privatization (I.e charter schools and vouchers)a s ‘choice.’
Reporters have “reformers” speed dial that provide a slanted inaccurate picture of what is happening. Public schools do not have paid “Hessian cheerleaders” and a team of marketeers in their corner.
If I may add a little to your thought: “Lazy reporters have. . . .”
And yes, you are quite correct with your thought! Thanks!
As always, Diane, thanks for posting about my work. I want to quickly respond to folks here who question why I still support standardized testing, even as I point out its many limitations.
We have a large and growing body of evidence that shows that school funding has a positive and significant effect on student outcomes. The reason we can say this is because researchers (including teams I have been a part of) are able to use testing outcomes to show a causal link between test scores and school funding.
We certainly don’t need to test every child in Grade 3-8 in two subjects to have adequate data to make this case. But we do need data: a good number of state-level school funding lawsuits have been built around the evidence test results provide.
I’ll try to blog more about this later. Again, thanks to Diane for highlighting my stuff, and thanks to all of you for reading it.
Your efforts and your research are appreciated, JJ. Personally, I don’t consider tests as much the problem as I do how they are (mis)used. Your endorsement of test-based accountability seems to be in keeping with the top priorities of Ed Reform. Nothing good occurs when these tests, even if they weren’t as flawed as you cogently point out, are used to punish students, teachers and/or schools. It’s not worth spending more time, effort, or money on improving these tests until that principle is established.
That “large body” of research? Where is it interred?
“researchers (including teams I have been a part of) are able to use testing outcomes to show a causal link between test scores and school funding.”
What is that “causal link”?
As it is, who gives a damn about completely invalid test scores to begin with? Obviously, you do Mark, as one must when doing one’s dissertation work with education statistics.
Have you read Noel Wilson’s work? If so, I’d appreciate any thoughts.
Because he has shown that your vaunted test score usage is as false and invalid as it comes. Start with garbage, garbage comes out. Calling standardized test scores garbage is being nice. Causal links using falsehoods can only net you falsehoods.
Love you, Jersey Jazzman, but you forget part of the point.
Your reasons for standardized testing ASSUME that those schools that get lower scores would get additional funding to help those students.
This is NOT THE CASE. “Failing schools” usually get less money and fewer resources, sometimes as punishment for “failing.” This happens all over the country.
You cannot make a pro-data argument when the feds and the states use the data to bash schools who need help.
That’s not completely true.
Many municpalities will spend more money to help such schools turn around. But if they don’t turn around within a certain amount of time, then they start to become defunded. Eventually, they stand to become privatized through charters.
But the problem with this scenario is that standarized tests should not, as in Finland and most of other countries, be used as the déterminant for success, and they are highly coorelated to SES of students. That does not mean that low SES students cannot excel on tests or in general, but one test per year to help dramatically shape the fate of the students, teacher and leadership, is a barbaric practice and policy.
“This is worth your while to read as you will learn a lot about standardized testing and its limitations.”
If you truly want to learn a lot about standardized testing and its complete invalidities (limitations), I suggest reading Noel Wilson’s never refuted nor rebutted “Educational Standards and the Problem of Error” found at: http://epaa.asu.edu/ojs/article/view/577/700
Brief outline of Wilson’s “Educational Standards and the Problem of Error” and some comments of mine. (updated 6/24/13 per Wilson email)
A description of a quality can only be partially quantified. Quantity is almost always a very small aspect of quality. It is illogical to judge/assess a whole category only by a part of the whole. The assessment is, by definition, lacking in the sense that “assessments are always of multidimensional qualities. To quantify them as unidimensional quantities (numbers or grades) is to perpetuate a fundamental logical error” (per Wilson). The teaching and learning process falls in the logical realm of aesthetics/qualities of human interactions. In attempting to quantify educational standards and standardized testing the descriptive information about said interactions is inadequate, insufficient and inferior to the point of invalidity and unacceptability.
A major epistemological mistake is that we attach, with great importance, the “score” of the student, not only onto the student but also, by extension, the teacher, school and district. Any description of a testing event is only a description of an interaction, that of the student and the testing device at a given time and place. The only correct logical thing that we can attempt to do is to describe that interaction (how accurately or not is a whole other story). That description cannot, by logical thought, be “assigned/attached” to the student as it cannot be a description of the student but the interaction. And this error is probably one of the most egregious “errors” that occur with standardized testing (and even the “grading” of students by a teacher).
Wilson identifies four “frames of reference” each with distinct assumptions (epistemological basis) about the assessment process from which the “assessor” views the interactions of the teaching and learning process: the Judge (think college professor who “knows” the students capabilities and grades them accordingly), the General Frame-think standardized testing that claims to have a “scientific” basis, the Specific Frame-think of learning by objective like computer based learning, getting a correct answer before moving on to the next screen, and the Responsive Frame-think of an apprenticeship in a trade or a medical residency program where the learner interacts with the “teacher” with constant feedback. Each category has its own sources of error and more error in the process is caused when the assessor confuses and conflates the categories.
Wilson elucidates the notion of “error”: “Error is predicated on a notion of perfection; to allocate error is to imply what is without error; to know error it is necessary to determine what is true. And what is true is determined by what we define as true, theoretically by the assumptions of our epistemology, practically by the events and non-events, the discourses and silences, the world of surfaces and their interactions and interpretations; in short, the practices that permeate the field. . . Error is the uncertainty dimension of the statement; error is the band within which chaos reigns, in which anything can happen. Error comprises all of those eventful circumstances which make the assessment statement less than perfectly precise, the measure less than perfectly accurate, the rank order less than perfectly stable, the standard and its measurement less than absolute, and the communication of its truth less than impeccable.”
In other words all the logical errors involved in the process render any conclusions invalid.
The test makers/psychometricians, through all sorts of mathematical machinations attempt to “prove” that these tests (based on standards) are valid-errorless or supposedly at least with minimal error [they aren’t]. Wilson turns the concept of validity on its head and focuses on just how invalid the machinations and the test and results are. He is an advocate for the test taker not the test maker. In doing so he identifies thirteen sources of “error”, any one of which renders the test making/giving/disseminating of results invalid. And a basic logical premise is that once something is shown to be invalid it is just that, invalid, and no amount of “fudging” by the psychometricians/test makers can alleviate that invalidity.
Having shown the invalidity, and therefore the unreliability, of the whole process Wilson concludes, rightly so, that any result/information gleaned from the process is “vain and illusory”. In other words start with an invalidity, end with an invalidity (except by sheer chance every once in a while, like a blind and anosmic squirrel who finds the occasional acorn, a result may be “true”) or to put in more mundane terms crap in-crap out.
And so what does this all mean? I’ll let Wilson have the second to last word: “So what does a test measure in our world? It measures what the person with the power to pay for the test says it measures. And the person who sets the test will name the test what the person who pays for the test wants the test to be named.”
In other words it attempts to measure “’something’ and we can specify some of the ‘errors’ in that ‘something’ but still don’t know [precisely] what the ‘something’ is.” The whole process harms many students as the social rewards for some are not available to others who “don’t make the grade (sic)” Should American public education have the function of sorting and separating students so that some may receive greater benefits than others, especially considering that the sorting and separating devices, educational standards and standardized testing, are so flawed not only in concept but in execution?
My answer is NO!!!!!
One final note with Wilson channeling Foucault and his concept of subjectivization:
“So the mark [grade/test score] becomes part of the story about yourself and with sufficient repetitions becomes true: true because those who know, those in authority, say it is true; true because the society in which you live legitimates this authority; true because your cultural habitus makes it difficult for you to perceive, conceive and integrate those aspects of your experience that contradict the story; true because in acting out your story, which now includes the mark and its meaning, the social truth that created it is confirmed; true because if your mark is high you are consistently rewarded, so that your voice becomes a voice of authority in the power-knowledge discourses that reproduce the structure that helped to produce you; true because if your mark is low your voice becomes muted and confirms your lower position in the social hierarchy; true finally because that success or failure confirms that mark that implicitly predicted the now self-evident consequences. And so the circle is complete.”
In other words students “internalize” what those “marks” (grades/test scores) mean, and since the vast majority of the students have not developed the mental skills to counteract what the “authorities” say, they accept as “natural and normal” that “story/description” of them. Although paradoxical in a sense, the “I’m an “A” student” is almost as harmful as “I’m an ‘F’ student” in hindering students becoming independent, critical and free thinkers. And having independent, critical and free thinkers is a threat to the current socio-economic structure of society.
“It’s no surprise that around 40 percent of students cleared the proficiency bar for the PARCC, and around 40 percent of adults in New Jersey have a bachelors degree.” — so, the test correlates to the reality? What a blow to test haters.
“I don’t know when we decided everyone should go to a four-year college.” – likewise, I don’t know why everyone should spend their time in high school in school barely learning basic algebra. Nine years is more than enough to squeeze the amount of knowledge presently delivered over thirteen years.
“By the time the school gets the score back, the student has already moved on to the next grade and another teacher.” – I’ve said the same before, but who am I. Here, Jersey Jazzman says it, this immediately gives this idea more weight.
PARCC was not designed to be a high school graduation test. No standardized test should be used for this purpose. Do you really want 60% of all students denied a high school diploma and unemployable except for manual labor?