New York State Allies for Public Education reviewed the recently released test scores and found curious anomalies. The group–which represents more than 50 other organizations–issued this statement.
http://www.nysape.org/nysape-pr-score-analysis.html
FOR IMMEDIATE RELEASE: August 5, 2016
NYS Allies for Public Education (NYSAPE) http://www.nysape.org
More information contact:
Lisa Rudley (917) 414-9190
Kevin Glynn (631) 291-2905
Bianca Tanis (845) 389-0722
Email: nys.allies@gmail.com
Did NYSED Manipulate Test Scores to Boost Proficiency?
Parents Demand Explanation of Anomalies in State Test Data and the Immediate Release of Suppressed Test Information
New data reveals that the percentage of raw points necessary to achieve a proficient performance (level 3) were lower on eleven out of the twelve 2016 NYS Common Core tests.
After analyzing the raw score to scaled score conversion charts that NYSED provides, advocates are asking whether NYSED has manipulated the raw score to scale score conversion in order to increase proficiency rates this year as the department did in the past under former NYS Commissioner of Education, Richard Mills.
While Commissioner Elia can factually claim that the “cut scores” have not changed this year, the percentage of raw points (also called raw scores) needed to receive a scale score associated with proficient performance were lowered across the board.
[Open link above to see graph Revised Cut Score Raw Percent]
An analysis provided by Michael O’Donnell of the New Paltz Board of Education as well as the chart below form the basis for concern.
[Again, open he link above for graph Percent of Raw Points Change.jpg]
Slight, annual changes in the raw scores required to achieve a given scale score are to be expected. This is a process called “equating,” which is routinely used in the administration and scoring of all standardized tests to reflect fluctuations in test difficulty. Compared to the implementation of Common Core-based state tests in 2013, however, the 2016 conversion charts show atypically large decreases in the raw scores required to be deemed proficient, especially in math. This could indicate that the 2016 tests were significantly more difficult than previous years. However, Commissioner Elia has repeatedly stated that the content of the 2016 tests was comparable to previous years in terms of rigor.
The use of test scores for high stakes accountability decisions makes test scores vulnerable to manipulation in order to serve political purposes. Maintaining the same “cut scores” from year to year while artificially decreasing the number of raw points needed achieve a scale score in the proficient performance range could result in inflated passing rates.
Given NYSED’s attempt to increase passing rates through the practice of untimed assessments and the State’s failure to maintain any data on the number, demographics, and performance of students who availed themselves of additional time, the state is admittedly unable to attribute or explain any increases in scores.
Michael O’Donnell, public school parent and New Paltz Board of Education member stated, “Assessment proficiency rates, in addition to not being reflective of college readiness or grade-level skills, are now not comparable to previous years’ results and have been subject to aggressive manipulation. It is hard to find any utility in these data.”
“The State seems to be doing everything it can to convince parents that these tests and the flawed standards they are based on are educationally sound. Year after year, 60 percent of our children are labeled as failing when we know this is simply not true. Increases in test scores based on inappropriate standards are meaningless, even more so if they have been manipulated to placate the public. We demand fully funded schools, equitable learning opportunities for our children, and an end to test and punish policies,” said Johanna Garcia, NYC parent and Co-President of District 6 President’s Council.
“It is foolish to have a conversation focused on data when 22% of students have opted-out. That is a 10% increase over the 20% who opted out last year. NYSED chose to cure flawed tests with fewer questions and unlimited time. When that wasn’t enough, they lowered the requirements for proficiency in a quest to show progress. Politicians gained new talking points, but students lost meaningful classroom time and schools are targeted for punishment because of flawed tests,” said Kevin Glynn, Long Island educator and public school parent.
Eileen Graham, Rochester public school parent and founder of Black Student Leadership said, “These exams have always negatively affected students, schools, and districts. As a black parent, I’m not satisfied with Commissioner Elia’s claims that the changes as they relate to black and Hispanic students are improvements or have created a different experience for this population. It seems she is using that narrative to appease and “trick” the community into believing things got better for the most vulnerable students. This is nonsense and in my opinion, patronizing.”
“The only question is how much the level of distortion goes up each year. Since Common Core-aligned testing began in 2013, SED and test publisher Pearson have depended on keeping information about the construction and quality of the exams hidden from independent review. Given all that has gone wrong over the last four years, it’s no wonder they want to operate in the dark,” said Fred Smith, testing specialist and former administrative analyst for New York City public schools.
“NYSAPE demands the public release of all state test analysis data from 2013-2016 and urges NYSED to account for the unprecedented lowering of the raw scores aligned with proficiency. These anomalies must be explained by Commissioner Elia if the State continues to maintain that state test scores are valid and reliable.
“NYSAPE is a grassroots coalition with over 50 parent and educator groups across the state.”
###
Diane Ravitch's blog 
Honestly, is anyone surprised. I’m just perplexed why the scores were stagnant last year. Perhaps we will once again be amazed after next year’s results when we witness once again how our students have “progressed” with the CCSS and the new improved assessments.
GI/GO
LikeLike
GI/GO or CI/CO or SI/SO all point to the inherent invalidities involved in the standards and testing malpractices. Why so many refuse to understand all of the onto-epistemological errors, falsehoods and psychometric fudging involved in those malpractices that render any results completely invalid is beyond me as Noel Wilson’s irrefutable work has been around for almost 20 years now:
“Educational Standards and the Problem of Error” found at: http://epaa.asu.edu/ojs/article/view/577/700
Brief outline of Wilson’s “Educational Standards and the Problem of Error” and some comments of mine.
1. A description of a quality can only be partially quantified. Quantity is almost always a very small aspect of quality. It is illogical to judge/assess a whole category only by a part of the whole. The assessment is, by definition, lacking in the sense that “assessments are always of multidimensional qualities. To quantify them as unidimensional quantities (numbers or grades) is to perpetuate a fundamental logical error” (per Wilson). The teaching and learning process falls in the logical realm of aesthetics/qualities of human interactions. In attempting to quantify educational standards and standardized testing the descriptive information about said interactions is inadequate, insufficient and inferior to the point of invalidity and unacceptability.
2. A major epistemological mistake is that we attach, with great importance, the “score” of the student, not only onto the student but also, by extension, the teacher, school and district. Any description of a testing event is only a description of an interaction, that of the student and the testing device at a given time and place. The only correct logical thing that we can attempt to do is to describe that interaction (how accurately or not is a whole other story). That description cannot, by logical thought, be “assigned/attached” to the student as it cannot be a description of the student but the interaction. And this error is probably one of the most egregious “errors” that occur with standardized testing (and even the “grading” of students by a teacher).
3. Wilson identifies four “frames of reference” each with distinct assumptions (epistemological basis) about the assessment process from which the “assessor” views the interactions of the teaching and learning process: the Judge (think college professor who “knows” the students capabilities and grades them accordingly), the General Frame-think standardized testing that claims to have a “scientific” basis, the Specific Frame-think of learning by objective like computer based learning, getting a correct answer before moving on to the next screen, and the Responsive Frame-think of an apprenticeship in a trade or a medical residency program where the learner interacts with the “teacher” with constant feedback. Each category has its own sources of error and more error in the process is caused when the assessor confuses and conflates the categories.
4. Wilson elucidates the notion of “error”: “Error is predicated on a notion of perfection; to allocate error is to imply what is without error; to know error it is necessary to determine what is true. And what is true is determined by what we define as true, theoretically by the assumptions of our epistemology, practically by the events and non-events, the discourses and silences, the world of surfaces and their interactions and interpretations; in short, the practices that permeate the field. . . Error is the uncertainty dimension of the statement; error is the band within which chaos reigns, in which anything can happen. Error comprises all of those eventful circumstances which make the assessment statement less than perfectly precise, the measure less than perfectly accurate, the rank order less than perfectly stable, the standard and its measurement less than absolute, and the communication of its truth less than impeccable.”
In other words all the logical errors involved in the process render any conclusions invalid.
5. The test makers/psychometricians, through all sorts of mathematical machinations attempt to “prove” that these tests (based on standards) are valid-errorless or supposedly at least with minimal error [they aren’t]. Wilson turns the concept of validity on its head and focuses on just how invalid the machinations and the test and results are. He is an advocate for the test taker not the test maker. In doing so he identifies thirteen sources of “error”, any one of which renders the test making/giving/disseminating of results invalid. And a basic logical premise is that once something is shown to be invalid it is just that, invalid, and no amount of “fudging” by the psychometricians/test makers can alleviate that invalidity.
6. Having shown the invalidity, and therefore the unreliability, of the whole process Wilson concludes, rightly so, that any result/information gleaned from the process is “vain and illusory”. In other words start with an invalidity, end with an invalidity (except by sheer chance every once in a while, like a blind and anosmic squirrel who finds the occasional acorn, a result may be “true”) or to put in more mundane terms crap in-crap out.
7. And so what does this all mean? I’ll let Wilson have the second to last word: “So what does a test measure in our world? It measures what the person with the power to pay for the test says it measures. And the person who sets the test will name the test what the person who pays for the test wants the test to be named.”
In other words it attempts to measure “’something’ and we can specify some of the ‘errors’ in that ‘something’ but still don’t know [precisely] what the ‘something’ is.” The whole process harms many students as the social rewards for some are not available to others who “don’t make the grade (sic)” Should American public education have the function of sorting and separating students so that some may receive greater benefits than others, especially considering that the sorting and separating devices, educational standards and standardized testing, are so flawed not only in concept but in execution?
My answer is NO!!!!!
One final note with Wilson channeling Foucault and his concept of subjectivization:
“So the mark [grade/test score] becomes part of the story about yourself and with sufficient repetitions becomes true: true because those who know, those in authority, say it is true; true because the society in which you live legitimates this authority; true because your cultural habitus makes it difficult for you to perceive, conceive and integrate those aspects of your experience that contradict the story; true because in acting out your story, which now includes the mark and its meaning, the social truth that created it is confirmed; true because if your mark is high you are consistently rewarded, so that your voice becomes a voice of authority in the power-knowledge discourses that reproduce the structure that helped to produce you; true because if your mark is low your voice becomes muted and confirms your lower position in the social hierarchy; true finally because that success or failure confirms that mark that implicitly predicted the now self-evident consequences. And so the circle is complete.”
In other words students “internalize” what those “marks” (grades/test scores) mean, and since the vast majority of the students have not developed the mental skills to counteract what the “authorities” say, they accept as “natural and normal” that “story/description” of them. Although paradoxical in a sense, the “I’m an “A” student” is almost as harmful as “I’m an ‘F’ student” in hindering students becoming independent, critical and free thinkers. And having independent, critical and free thinkers is a threat to the current socio-economic structure of society.
LikeLike
Duane, you are starting to convince me.
My grand daughter entering ninth grade has just about given up on school while my son with a GED is convinced he needs to be perfect to go to college and he doesn’t believe he will be able to meet those expectations.
Let’s encourage learning for the joy of learning instead of studying to participate in worthless tests.
LikeLike
Glad to see you coming around flos56!
If you haven’t read Wilson I highly recommend it as my summary doesn’t do justice to the entire work (even with Noel reviewing/correcting my summary).
If you read it and have any questions about it feel free to contact me at dswacker@centurytel.net and I will help you with any questions you may have. Just reference “Wilson’s work” in the subject line.
LikeLike
“This is a process called “equating,” which is routinely used in the administration and scoring of all standardized tests to reflect fluctuations in test difficulty.”
It’s what Wilson calls “fudging” and that word doesn’t have a positive connotation.
LikeLike
Eileen Graham, Rochester public school parent and founder of Black Student Leadership said, “These exams have always negatively affected students, schools, and districts. As a black parent, I’m not satisfied with Commissioner Elia’s claims that the changes as they relate to black and Hispanic students are improvements or have created a different experience for this population. It seems she is using that narrative to appease and “trick” the [minority] community into believing things got better for the most vulnerable students. This is nonsense and in my opinion, patronizing.”
BINGO!
Glad to see that adults in the minority community are finally realizing that modern, test-and-punish reform and the charter movement it enabled, have been nothing more than a shameless SCAM, preying on the desperation of parents trapped in the hopeless grip of generational poverty and dependence.
LikeLike
What a mess!
LikeLike