Leonie Haimson, founder of Class Size Matters, is a major figure in New York City education circles.
She wrote this post about the reporting of the state test scores. First, the public learned that the test scores were up. Next, those who bothered to read Commissioner Elia’s statement that accompanied the release learned that this year’s scores were not comparable to previous years’ scores. But then they saw Commissioner Elia and the media celebrating the test score gains that were not comparable to previous years.
What you will learn from her post is that there is almost always a political slant in reporting scores, especially these days, when so many politicians want to claim credit for rising scores. If officials want the scores to look good, they will magnify gains. Or they can change the passing mark to create artificial gains. Or they can convert raw scores to higher scores. When they first get started, they want the scores as low as possible so they can claim gains afterwards.
Leonie provides the historical context that was absent from reporting on this year’s scores. We have seen this play before. We saw the scores go up and up and up from 2002-2009. Then an independent team of researchers studied the tests and the state acknowledged score inflation. And the scores came crashing down. But not until after Mayor Bloomberg was safely re-elected to a third term, based in large part on the historic improvement in test scores (that disappeared in 2010).
It is time to admit that the scores are malleable. What do they represent? One thing for sure is that the kids with the advantages are always at the top, and the kids without the advantages are always at the bottom. No matter how often we test, no matter what the test, the results are unchanged year after year.
Maybe it is time to step back from the incessant testing and to focus instead on interventions that might change the life chances for children and the educational outcomes as well.
Diane Ravitch's blog 
Diane, not only are the scores malleable, but it looks like the tests are as well: https://exceptionaldelaware.wordpress.com/2016/08/08/the-high-stakes-testing-scam-revealed-at-last/
The line “It is time to admit that the scores are malleable” caught my attention also.
But it doesn’t matter if the scores are malleable, of course they are, it’s part of the psychometric fudging involved in the standards and testing regime that, along with all the onto-epistemological errors and falsehoods, render any interpretation of the results/scores COMPLETELY INVALID. To understand that complete invalidity read and understand Noel Wilson’s never refuted nor rebutted total take down of those two malpractices in his “Educational Standards and the Problem of Error” found at:
http://epaa.asu.edu/ojs/article/view/577/700
Brief outline of Wilson’s “Educational Standards and the Problem of Error” and some comments of mine.
1. A description of a quality can only be partially quantified. Quantity is almost always a very small aspect of quality. It is illogical to judge/assess a whole category only by a part of the whole. The assessment is, by definition, lacking in the sense that “assessments are always of multidimensional qualities. To quantify them as unidimensional quantities (numbers or grades) is to perpetuate a fundamental logical error” (per Wilson). The teaching and learning process falls in the logical realm of aesthetics/qualities of human interactions. In attempting to quantify educational standards and standardized testing the descriptive information about said interactions is inadequate, insufficient and inferior to the point of invalidity and unacceptability.
2. A major epistemological mistake is that we attach, with great importance, the “score” of the student, not only onto the student but also, by extension, the teacher, school and district. Any description of a testing event is only a description of an interaction, that of the student and the testing device at a given time and place. The only correct logical thing that we can attempt to do is to describe that interaction (how accurately or not is a whole other story). That description cannot, by logical thought, be “assigned/attached” to the student as it cannot be a description of the student but the interaction. And this error is probably one of the most egregious “errors” that occur with standardized testing (and even the “grading” of students by a teacher).
3. Wilson identifies four “frames of reference” each with distinct assumptions (epistemological basis) about the assessment process from which the “assessor” views the interactions of the teaching and learning process: the Judge (think college professor who “knows” the students capabilities and grades them accordingly), the General Frame-think standardized testing that claims to have a “scientific” basis, the Specific Frame-think of learning by objective like computer based learning, getting a correct answer before moving on to the next screen, and the Responsive Frame-think of an apprenticeship in a trade or a medical residency program where the learner interacts with the “teacher” with constant feedback. Each category has its own sources of error and more error in the process is caused when the assessor confuses and conflates the categories.
4. Wilson elucidates the notion of “error”: “Error is predicated on a notion of perfection; to allocate error is to imply what is without error; to know error it is necessary to determine what is true. And what is true is determined by what we define as true, theoretically by the assumptions of our epistemology, practically by the events and non-events, the discourses and silences, the world of surfaces and their interactions and interpretations; in short, the practices that permeate the field. . . Error is the uncertainty dimension of the statement; error is the band within which chaos reigns, in which anything can happen. Error comprises all of those eventful circumstances which make the assessment statement less than perfectly precise, the measure less than perfectly accurate, the rank order less than perfectly stable, the standard and its measurement less than absolute, and the communication of its truth less than impeccable.”
In other words all the logical errors involved in the process render any conclusions invalid.
5. The test makers/psychometricians, through all sorts of mathematical machinations attempt to “prove” that these tests (based on standards) are valid-errorless or supposedly at least with minimal error [they aren’t]. Wilson turns the concept of validity on its head and focuses on just how invalid the machinations and the test and results are. He is an advocate for the test taker not the test maker. In doing so he identifies thirteen sources of “error”, any one of which renders the test making/giving/disseminating of results invalid. And a basic logical premise is that once something is shown to be invalid it is just that, invalid, and no amount of “fudging” by the psychometricians/test makers can alleviate that invalidity.
6. Having shown the invalidity, and therefore the unreliability, of the whole process Wilson concludes, rightly so, that any result/information gleaned from the process is “vain and illusory”. In other words start with an invalidity, end with an invalidity (except by sheer chance every once in a while, like a blind and anosmic squirrel who finds the occasional acorn, a result may be “true”) or to put in more mundane terms crap in-crap out.
7. And so what does this all mean? I’ll let Wilson have the second to last word: “So what does a test measure in our world? It measures what the person with the power to pay for the test says it measures. And the person who sets the test will name the test what the person who pays for the test wants the test to be named.”
In other words it attempts to measure “’something’ and we can specify some of the ‘errors’ in that ‘something’ but still don’t know [precisely] what the ‘something’ is.” The whole process harms many students as the social rewards for some are not available to others who “don’t make the grade (sic)” Should American public education have the function of sorting and separating students so that some may receive greater benefits than others, especially considering that the sorting and separating devices, educational standards and standardized testing, are so flawed not only in concept but in execution?
My answer is NO!!!!!
One final note with Wilson channeling Foucault and his concept of subjectivization:
“So the mark [grade/test score] becomes part of the story about yourself and with sufficient repetitions becomes true: true because those who know, those in authority, say it is true; true because the society in which you live legitimates this authority; true because your cultural habitus makes it difficult for you to perceive, conceive and integrate those aspects of your experience that contradict the story; true because in acting out your story, which now includes the mark and its meaning, the social truth that created it is confirmed; true because if your mark is high you are consistently rewarded, so that your voice becomes a voice of authority in the power-knowledge discourses that reproduce the structure that helped to produce you; true because if your mark is low your voice becomes muted and confirms your lower position in the social hierarchy; true finally because that success or failure confirms that mark that implicitly predicted the now self-evident consequences. And so the circle is complete.”
In other words students “internalize” what those “marks” (grades/test scores) mean, and since the vast majority of the students have not developed the mental skills to counteract what the “authorities” say, they accept as “natural and normal” that “story/description” of them. Although paradoxical in a sense, the “I’m an “A” student” is almost as harmful as “I’m an ‘F’ student” in hindering students becoming independent, critical and free thinkers. And having independent, critical and free thinkers is a threat to the current socio-economic structure of society.
I hope the memo reaches two-tier Roland Fryer. The Deutsch 29 blog, posted the “philanthropic’ grants on his c.v. and, his comments about, what kids like his, need, versus what other people’s kids need.
I wish Leonie or someone else would address how the top performing charter schools – the ones whose scores significantly bump up the “average” of charter schools – have achieved “rising scores” on state tests by having mysteriously shrinking class sizes.
One Success Academy school — Bed Stuy 1 — had only 2 5th graders who scored below proficient this year on the ELA. Last year in 4th grade there were 16 students below proficient. That would seem impressive until you realize that a whopping 17 students (23% of the class) disappeared from the cohort after 4th grade testing.
In fact, when you look at the number of “proficient” students last year in the 4th grade (57) and the number this year in 5th grade (54), the number of proficient students declined! But because so many students disappeared — anyone want to lay bets on how many were the non-proficient ones? — it suddenly appears that the school experienced a 20% increase in proficiency rates.
As long as those proficiency rates are looked at by this warped “overall” method instead of seeing what is happening to each child who enrolls in the school in Kindergarten and how they progress through 5th grade, does it really matter how cut scores are developed and what % are proficient?
We have a system where the schools practicing the most reprehensible weeding out practices will be rewarded the most. Until that is addressed, these rates are meaningless.
^^^and one reason I would think that Leonie should be interested is because this kind of false “improvement” and “performance” is used by politicians to justify NOT having small class sizes! Eva Moskowitz herself has stated for the record that small class size is a luxury and her results prove beyond a shadow of a doubt that is is completely unnecessary. If she can get that improvement at Bed Stuy 1, why can’t every public school?
Until that myth that Eva Moskowitz and her billionaire supporters have promoted is finally shown to be false, class size will be assumed NOT to matter. And the only way to address these kind of outrageous misleading statements is to show that the people promoting them are liars with their own agenda that has nothing to do with what is best for all students.
The largest retailer of schools-in-a-box (co-owned by Bill Gates), was founded based on a “scale”, reached with about 60 students per class. A 20% return, for the schools-in-a-box business model, was cited by BIA’s founder. New Schools Venture Fund ($22 mil. in Gates Foundation funding) describes the “marching orders…to develop diverse charter school organizations that produce different brands on a large scale.”
The Aspen Institute’s education programs get funding from the Gates
Foundation. Aspen brags about its success in getting its plans implemented. One of Aspen’s programs is the Senior Congressional Education Staff Network. IMO, Aspen has been very successful in transmuting public education to which I reply, they are our schools, our children and it is our taxes. Reformers want good schools for their kids and so do we. Don’t use politicians to rip off the productive Americans that fund your lavish lifestyles.
I cross posted this at Oped http://www.opednews.com/Quicklink/NYC-Public-School-Parents-in-Best_Web_OpEds-Corporate-Fraud_Diane-Ravitch_Fraud_Media-160809-819.html
with commentary that has embed links at the site.
And look at Denver and see what the tests are meant to do…
Denver: 47 Teachers Will Lose Their Tenure Because of Ratings System Based 50% on Test Scores .
Long before the testing ploy was created to rid the schools of REAL TEACHERS, LA blacklisted teachers andfabricated charges, to silence the voices of tens of thousands of real educators who would never use the Gates common core crap.
Testing is just the latest PROCESS to get rid of REAL PROFESSIONAL EDUCATORS. Step one was in the nineties, when this happened in NYC, http://www.perdaily.com/2011/01/lausd-et-al-a-national-scandal-of-enormous-proportions-by-susan-lee-schwartz-part-1.html so that public schools would fail
and THIS was the result
http://gemnyc.org/2012/05/20/the-inconvenient-truth-behind-waiting-for-superman-now-online/
This is third post on the subject of the release of test scores by the NYSDOE.
First time you stated that the scores were released on a late Friday evening, which is a dead time for news releases. It does not get any mainstream media mention on Friday and by Monday everything is forgotten and it gets no exposure. You accused that it was done by NYSDOE with all due deliberation.
The second post on this subject you cited a blog post that the test data was manipulated to suit their slant by NYSDOE without any proof.
This third post once again cites another blog which takes the conspiracy theory one step further to include political slant in reporting scores. This blog entry insinuates political gains by changing cut scores (passing score) to magnify gains, artificial gains, manipulation of raw scores etc. This blog entry claims that the test scores are malleable to suit the powers to be in the NYSDOE.
This entry claims, “No matter how often we test, no matter what the test, the results are unchanged year after year.” There is no proof of this statement. This must mean that the kids, teachers and administrators have not changed for the past 15 years in spite of the education reform bias that you claim so often to be cause of all the troubles in the education sector. You also claim that poverty has increased during the same period and it did not make much of an impact on test scores.
I have an idea for your fourth post. You can bring in the NCLB and RTTT slant for all these shenanigans performed by the NYSDOE.
Raj,
All the talk of test scores and the accompanying BS is all just mental masturbation to begin with. When one starts with error and falsehoods as the test results are, why even talk about them as any conclusions drawn are bound to be error- and falsehood-ridden. So little time so much bloviation about what in essence is utter nonsense.
Raj, I have an idea for your next 12 posts:
You can state for the record that what you admire the most is how high a school can get their proficiency rates on these exams. Even if it means a school has fewer and fewer proficient students as the cohort ages, as long as the low-scoring kids disappear, you are inordinately impressed. No doubt you want all schools to follow the “weeding out” method so they can get the proficiency results that impresses you to no end.
I am surprised this is what you teach students — that it’s always the ratio of passing to non-passing students that is a measure of a school and the school that does an excellent job weeding out those students is one that we should all admire the most. But then again, I am sure you are well-paid for promoting that view. I am just glad you aren’t a scientist who is doing a study of drugs with such an outrageously idiotic methodology.
The Raj study of a new chemotherapy drug: 100 cancer patients started a new chemo drug invented by a “charter hospital” where it is tested “randomly”. Three years later, according to Raj, 100% of the patients did not experience any recurrence of the cancer while on that drug. Of course, since it was done in a “charter hospital”, 60 of those patients were no longer there at the end of the three years. And we have no idea if they got much worse from the newly designed “charter drug”. But Raj says that what is important is that the drug works 100% of the time according to the kind of studies he admires the most.
Always dump the kids that hurt your ability to brag about proficiency rates and no doubt scholars like Raj will insist that we should take your passing rates very seriously and all try to copy you.
NYC public school parent
I never said anything about testing students.
I never said anything about the ratio between passing or not passing the exams.
I never said anything about disappearing students.
I never studied chemotherapy or charter hospitals (they do not exist) or charter schools.
I know that NYSDOE released test data and Diane Ravitch has problems accepting the fact that NYSDOE released the data. They had no choice in the matter. They were just following state law.
If you have any problems with the data please deal with NYSDOE.
I do not see how you read between the lines to guess what I am thinking. Please read exactly what I said in my comment.
Just to be very clear, please remember that I also never said anything about the following:
public schools,
private schools,
charter schools,
Success Academy,
Failure Academy (rebranding of all the failing public schools), oligarchs,
Arne Duncan,
John King,
Obama,
Koch brothers,
Bill Gates,
Eli Broad,
Eva Moskovitz,
John Deasy,
Commissioner Elia and many more.
I’d like the next blog post, to cite the salaries and monetary gains that all of the leeches, blood sucking from public school transmutation, have pocketed, at the expense of students, communities and taxpayers.
Raj, you expressed your anger that Diane Ravitch was critical of these exams and the people who misuse the results for nefarious agendas.
You accused Diane Ravitch of being a conspiracy theorist because she expressed doubts about the usefulness of these cut scores.
Now you pretend that you aren’t saying these tests are valuable at all. But then why would you be so angry that Diane criticized them?
Let me give you an example: people who angrily attack critics of Donald Trump are pro-Trump. You can’t turn around and say “I’m not saying that he is an honest and trustworthy person and I’m not saying that even half of what comes out of his mouth is truthful, and I’m not agreeing with him when he insults Muslims and women. I’m just angrily attacking anyone who dares to question Mr. Trump but I have no good reason for doing so except my desire to defend Mr. Trump from any critics.”
You sound just as ridiculous when you claim you don’t have great respect for the exams and great respect for any charter school with high passing rates regardless of how many children are shed along the way. You adore them because without them charters would be revealed as no better — and usually worse — than public schools. You desperately need to average in large charter networks who are expert at making those low-performing kids disappear while convincing their parents it is the kid (at age 6!) who is the problem, not the school. Nice, right? You must be really delighted to think about how that might have impacted those children’s lives.
Since you didn’t defend the tests from my criticisms. no doubt you agree with me or you don’t agree with me but have nothing to defend your love of testing except for the fact that it helps the people who want to privatize public education and attack public school teachers.
Either you believe that the tests are beyond reproach and that’s why you are so angry at Diane Ravitch for criticizing them, or you agree with her that the tests are flawed measures of a school. Since it is obvious from your angry criticism that you think the tests are beyond reproach, just admit it instead of pretending you have no opinion, just need to angrily attack any critics anyway!
Trends in cut scores here: http://www.schoolmeter.com/sm/reports/ela_math_scores_2011_2016a.pdf