Jere Hochman superintendent of the Bedford Central School District in New York, tries to make sense of the latest test results in this blog.
He has a series of spot-on metaphors.
The state’s policy is based, he writes, is best described as “Fire. Aim. Ready.”
He adds:
“Raising the bar? High expectations? Every student means every? Rigorous standards? Benchmark assessments? No problem. But don’t make kids and teachers collateral damage due to accelerated, unmapped, make-up-the-rules-as-you-go-along implementation.”
What is it like to be controlled by a state education department that makes up the rules as it goes along, turning students, teachers, and administrators into collateral damage for their half-baked plans and policies?
Diane Ravitch's blog 
Think how similar this is to our economic debate that has been dominated by Very Serious People (as Krugman calls them), who stroke their chins and talk about hard facts and sacrifice (always only affecting other people). All this Seriousness hides their lack of seriousness in the form of actual data, workable theories, and rigorous methods for determining what works and what doesn’t. It also serves as a cover for serving the interests of a few while hurting the many.
LikeLike
Bingo!
LikeLike
When you “build the airplane as it is flying,” you end up crashing in the Andes and eating each other.
LikeLike
When I complained to the district special ed. people about how the new, untested and untried CC standards were affecting my son with a math disability, one of the special ed. directors stated that they were “building the airplane as it was flying.” It was so flippant, because these are kids’ LIVES they’re playing with. I was really disgusted.
LikeLike
Six word response: My lawyer will be contacting you!
LikeLike
Could you/did you opt him out of the test?
My current line of thinking is that sense cannot be made out of these tests that I am nicknaming “weapons of mass destruction”.
LikeLike
I am not recommending what NY citizens should do; just pointing it out.
Petitioning New York State Board of Regents
New York State Board of Regents: Terminate the Employment of State Education Commissioner, John King
Petition by
Coalition for Justice in Education
LikeLike
Perhaps Jere Hochman missed the fact that the state is not using this year’s data to punish schools (no new Focus Schools identified) or professionals (encouraging districts not put APPR on hold).
What’s Jere Hochman’s plan to move forward? Same old business as usual?
It’s easy to criticize, but it’s harder to lead.
LikeLike
Even if they aren’t sued this year (and I wouldn’t trust that any further than I can throw it), it WILL be used in the next year or so. What’s to say that the scores will be any better? Maybe even worse? Good teachers, especially those who teach at-risk populations, WILL lose their jobs over this, if something isn’t done.
LikeLike
I believe he does both…what do you do? Talk about same old.
LikeLike
It’s also hard to think when one is so credulous concernng people who lie about everything.
Then again, as a charter school operator, you’re the beneficiary of their lies, so your credulity serves your interests.
Funny how consistently that pattern repeats among so-called reformers.
LikeLike
I posted 19 suggestions in the blog
http://thinkingaboutschools-jhstlny.blogspot.com/
and here’s what we are doing.
When I am asked, “What do you think about the scores?” my response is generally, “For which purpose?” It’s not just about a score and we ask many questions:
What do the scores tell us about curriculum alignment, planning, and time? We analyze…
• District-wide, school, and district grade level performance by subject for alignment and gaps
• Item analyses and particular strands of test items/topics for alignment and gaps
• Performance in relation to planning units of study, pedagogy, and routine class assessments
What do the scores tell us about school performance? We analyze
• A school’s performance overall and by subject
• “Outlier” classrooms, grade-levels, and schools
What do the scores tell us about students? We analyze…
• Performance of disaggregated groups of students
• Performance of each student overall and by item analysis
LikeLike
Jere,
“When I am asked, “What do you think about the scores?” my response is generally, “For which purpose?” It’s not just about a score and we ask many questions: What do the scores tell us about curriculum alignment, planning, and time?. . . What do the scores tell us about school performance?. . . What do the scores tell us about students? We analyze…”
Actually the scores tell you absolutely nothing that has any basis in logical thought as the scores are derived from a process that is so filled with logical errors that any conclusions are completely invalid. The scores and your all’s attendant analysis are chimeras, duendes, wisp o nothings or as Wilson puts it “vain and illusory”. To understand why read, understand and learn his “Educational Standards and the Problem of Error” found at: http://epaa.asu.edu/ojs/article/view/577/700
Sancho, ¡ayúdame, por favor! ¡Qué estos sabe de todos entiendan los daños que hacen a los inocentes con sus locuras de prácticas educativos!
Brief outline of Wilson’s “Educational Standards and the Problem of Error” and some comments of mine. (updated 6/24/13 per Wilson email)
1. A quality cannot be quantified. Quantity is a sub-category of quality. It is illogical to judge/assess a whole category by only a part (sub-category) of the whole. The assessment is, by definition, lacking in the sense that “assessments are always of multidimensional qualities. To quantify them as one dimensional quantities (numbers or grades) is to perpetuate a fundamental logical error” (per Wilson). The teaching and learning process falls in the logical realm of aesthetics/qualities of human interactions. In attempting to quantify educational standards and standardized testing we are lacking much information about said interactions.
2. A major epistemological mistake is that we attach, with great importance, the “score” of the student, not only onto the student but also, by extension, the teacher, school and district. Any description of a testing event is only a description of an interaction, that of the student and the testing device at a given time and place. The only correct logical thing that we can attempt to do is to describe that interaction (how accurately or not is a whole other story). That description cannot, by logical thought, be “assigned/attached” to the student as it cannot be a description of the student but the interaction. And this error is probably one of the most egregious “errors” that occur with standardized testing (and even the “grading” of students by a teacher).
3. Wilson identifies four “frames of reference” each with distinct assumptions (epistemological basis) about the assessment process from which the “assessor” views the interactions of the teaching and learning process: the Judge (think college professor who “knows” the students capabilities and grades them accordingly), the General Frame-think standardized testing that claims to have a “scientific” basis, the Specific Frame-think of learning by objective like computer based learning, getting a correct answer before moving on to the next screen, and the Responsive Frame-think of an apprenticeship in a trade or a medical residency program where the learner interacts with the “teacher” with constant feedback. Each category has its own sources of error and more error in the process is caused when the assessor confuses and conflates the categories.
4. Wilson elucidates the notion of “error”: “Error is predicated on a notion of perfection; to allocate error is to imply what is without error; to know error it is necessary to determine what is true. And what is true is determined by what we define as true, theoretically by the assumptions of our epistemology, practically by the events and non-events, the discourses and silences, the world of surfaces and their interactions and interpretations; in short, the practices that permeate the field. . . Error is the uncertainty dimension of the statement; error is the band within which chaos reigns, in which anything can happen. Error comprises all of those eventful circumstances which make the assessment statement less than perfectly precise, the measure less than perfectly accurate, the rank order less than perfectly stable, the standard and its measurement less than absolute, and the communication of its truth less than impeccable.”
In other word all the errors involved in the process render any conclusions invalid.
5. The test makers/psychometricians, through all sorts of mathematical machinations attempt to “prove” that these tests (based on standards) are valid-errorless or supposedly at least with minimal error [they aren’t]. Wilson turns the concept of validity on its head and focuses on just how invalid the machinations and the test and results are. He is an advocate for the test taker not the test maker. In doing so he identifies thirteen sources of “error”, any one of which renders the test making/giving/disseminating of results invalid. As a basic logical premise is that once something is shown to be invalid it is just that, invalid, and no amount of “fudging” by the psychometricians/test makers can alleviate that invalidity.
6. Having shown the invalidity, and therefore the unreliability, of the whole process Wilson concludes, rightly so, that any result/information gleaned from the process is “vain and illusory”. In other words start with an invalidity, end with an invalidity (except by sheer chance every once in a while, like a blind and anosmic squirrel who finds the occasional acorn, a result may be “true”) or to put in more mundane terms shit in-crap out.
7. And so what does this all mean? I’ll let Wilson have the second to last word: “So what does a test measure in our world? It measures what the person with the power to pay for the test says it measures. And the person who sets the test will name the test what the person who pays for the test wants the test to be named.”
In other words it measures “’something’ and we can specify some of the ‘errors’ in that ‘something’ but still don’t know [precisely] what the ‘something’ is.” The whole process harms many students as the social rewards for some are not available to others who “don’t make the grade (sic)” Should American public education have the function of sorting and separating students so that some may receive greater benefits than others, especially considering that the sorting and separating devices, educational standards and standardized testing, are so flawed not only in concept but in execution?
My answer is NO!!!!!
One final note with Wilson channeling Foucault and his concept of subjectivization:
“So the mark [grade/test score] becomes part of the story about yourself and with sufficient repetitions becomes true: true because those who know, those in authority, say it is true; true because the society in which you live legitimates this authority; true because your cultural habitus makes it difficult for you to perceive, conceive and integrate those aspects of your experience that contradict the story; true because in acting out your story, which now includes the mark and its meaning, the social truth that created it is confirmed; true because if your mark is high you are consistently rewarded, so that your voice becomes a voice of authority in the power-knowledge discourses that reproduce the structure that helped to produce you; true because if your mark is low your voice becomes muted and confirms your lower position in the social hierarchy; true finally because that success or failure confirms that mark that implicitly predicted the now self evident consequences. And so the circle is complete.”
In other words students “internalize” what those “marks” (grades/test scores) mean, and since the vast majority of the students have not developed the mental skills to counteract what the “authorities” say, they accept as “natural and normal” that “story/description” of them. Although paradoxical in a sense, the “I’m an “A” student” is almost as harmful as “I’m an ‘F’ student” in hindering students becoming independent, critical and free thinkers. And having independent, critical and free thinkers is a threat to the current socio-economic structure of society.
LikeLike
educativAs
LikeLike
http://comsewogue.org/news.cfm?story=123780&school=0
Another Superintendent speaking truth to power!
LikeLike
Powerful. I hope more Superintendents follow his lead.
LikeLike
Jere Hochman’s points are right on target. As a teacher, I know first-hand the effects of the “make it up as you go” policies that have been coming down the pike for many years now. One good example I will cite as evidence… for the past 3 years (and there will be once again a change this year), my department head has held a meeting at the start of the year which requires us to completely overhaul the format of our lesson plans (which of course includes changing how we organize the teaching of our subject). Why? Because we are required to meet the national standards in order to qualify for federal money and must be able to “prove” we are following the standards. So for three years running and a 4th upcoming, there is a new and improved lesson plan format which effects how the lessons are taught in the class. So this year, the “catch word” is SLO… student learning objectives… and this is going to require me to provide a “measurable” break-down of statistics on the student body of each and every class I teach and to “measure” the learning with data of each and every student (being that I am a specialist who will work in several schools this year… all I can say is huh????). Mind you the student population is always in flux in title one schools where I teach. I will also have to establish a “base-line” lesson in order to be able to measure “before” and “after” learning. Given that I will see these students a rather limited amount of times, establishing a “base-line” before each real lesson WILL BECOME THE LESSON! Once again.. huh???? A simple warm-up activity to engage them will not do as it doesn’t fit squarely into DATA ACCOUNTABILTY. When data becomes the end all, we have to start questioning the detrimental effects of this national “make it up as you go” policy we have been forced to abide by for way too long. Our nation’s children deserve better! @kitchensinkalive…the problem is that for many years now it has been BUSINESS as usual… all business and no education.
LikeLike
Reblogged this on David R. Taylor-Thoughts on Texas Education.
LikeLike
Mr. Hochman,
Thank you for your recent statement. Your metaphor could not have been more perfect.
As I know you’re likely to be reading this, it’s now up to you and your board to systemically hold (or continue to hold) frequent open board meetings and public hall meetings in which you and the board can explain very explicitly to parents how NYSED has caused this current train wreck and wreaked demoralization upon our families, students, and educators.
I just received a letter of support and acknowledgement from my own supernintendent, basically confirming that while we have to move forward with CCSS, we as an LEA have absolutely NOT failed our students in the context of these newly released junk science scores in ELA and math . . . . I was surprised to have received this letter because my superintendent is quite young and almost brand new at his job. Yet, I was delighted and empowered upon absorbing his findings, and I commend him and all other high level public education executives for corrorborating what teachers have discovered and for speaking out about how state and federal governments are creating, implementing, enforcing and dealing with the policies they have generated.
I hope you will use more than mainly newletters, e-blasts, connect-eds, and social media in Bedford to educate parents about what is really going on in this reform movement. I am not saying we teachers can simply have everything we want in our contracts, but what I am positing is that live, face-to-face meetings with people in the flesh and blood are often the most powerful way to impart truth and empower all stakeholders’ thinking.
I am hopeful you will continue as such.
LikeLike
Robert,
You are quite correct to assert that it is incumbent upon those in power to frequently, forcefully, loudly and logically decry these educational malpractices and show how not doing them is a much wiser course of action.
But I won’t hold my breath! Haven’t seen much of it yet except for a few bold individuals. Most prefer to give up their logical thought capabilities to the supposed experts and/or powers that be. Sheople?
LikeLike
You meant “Sheeple”, I think.
Superintendents are multi-taskers, working 24/7.
Not that we teachers aren’t!
I am as much a carmudgeon as the next guy, (you included?). At the very least, I am an independent thinker and a true cynic about mankind in general.
But let’s encourage Mr. Hochman in this instance. He has come this far, and I really believe he means what he says because he is a stakeholder as are the principals and teachers under him.
I am no optimist, but in some choice instances, we may get more done seeing the Hochman glass as half full. I work in the district about 20 minutes by car away from his. In fact, I interviewed at his district about 11 years ago.
Y exactamente quien es “Sancho” en tu escritura por encima de este commentario? Estoy curioso, si no te molesta que te pido . . .
BTW, thank you for your note about your whereabouts in MO. Chesterfield is indeed not so far from where you are. The rivers in MO are amazing! They are like worlds unto themselves. They wind and snake about. .. their scale and layout are true miracles. So beautiful!
But I have to say that Chesterfield as a suburb, an upper middle class one at that, was beautiful, peaceful, idyllic, organized, well facilitated, sparkling clean, and incredibly friendly. You could hear a pin drop.
Which is why I was bored to tears, in that particular enclave of bland, white bread, vanilla, almost Stepford residents. Everyone was so non-opinionated (unless you painted your house the wrong color, like periwinkle), smiling, coming to your house with a bundt cake, grilling eggplant . . . . the homes starting at 3,000 square feet on their two to three acre lots. . . the cars always looking as though they just came off the lot. All the guys unwinding after work in their I-zods and even the jogging women wore pearls with sweat pants.
Is is me, or am I just a snooty North Easterner who likes the grit of the city or the melange of a good mill town gone extinct but trying to survive?
Don’t mean to digress, but I did like the botanical gardents in St. Lous as well as that amazing white arch . . .
LikeLike
Robert,
I do indeed excoriate those in positions of power on these matters. It’s good that there are others who are against these educational malpractices that are more optimistic (Diane for sure) and less cynical and caustic like myself. I do need to be reminded that one is more likely to succeed through gentle persuasion than through “rigor”. But I find that the edudeformers also have the art of gentle persuasion (advertising/propaganda/media control) down pat and sometimes people need to be shocked to their senses. I’m trying to goad the GAGA (go along to get along) group into coming to their sentences and yes, I can be quite gruff in doing so. I reserve that right as an older (not as old as Dick V) grumpy fart Spanish teacher.
Your description of Chesterfield is really spot on! Your perception was excellent but your memory of the Arch is a little shaky. It is sheathed in polished aluminum and therefore is a shiny silver color not white but I can see why you remember it as white because on a really sunny day with the sun glinting off it it takes on a silvery white tone. If it was made in ancient times, especially in the Western hemisphere it probably would have been sheathed in gold as the sun was considered the giver of life.
Sancho = Sancho Panza of “Don Quixote de la Mancha”, DQ’s faithful squire who is caught between reality and irreality in DQ’s escapades. I have been referencing (hopefully amusingly) this novel’s characters as a means of highlighting my Quixotic Quest to rid the world of educational standards, standardized testing and the grading of students.
And thanks for the reference to eggplants. I made up and froze a bunch eggplant parmesan “patties” and need to put them in plastice bags. When the produce starts coming on it gets busy. Today, we have to make salsa with the 150 lbs of tomatoes we picked the last couple of days (with another couple of hundred ready to be picked) and make another (yes another) 2-3 dozen pickles with the cucumbers we have picked, again with that many more on the plants. Pole beans need picking. Slimy okra is probably ready. And what’s left of our sweet corn (we’ve already trapped and dispatched with three coons) is about ready to be picked. Just waiting for the peppers and various other crops. 30 years ago Chesterfield used to be more like where I live now-farms, fields and woods.
LikeLike
Dr. Theresa Thayer Snyder from the Voorheesville School District in New York has a powerful letter on the Superintendents Page of the school website. It’s the second one I’ve read today. I am thrilled local leadership is finally recognizing the damage being done by our State Education Department and finding the courage to speak out. Those brave enough to stand up for our children deserve our applause and recognition! Bravo to them all!
LikeLike
Can you post a link to her letter?
LikeLike
http://vcsd.neric.org/superintendent/superintendent.htm
LikeLike
Excerpt:
Our community is sophisticated enough to recognize a canard when it experiences one. These tests were intentionally designed to obtain precisely the outcomes that were rendered. The rationale behind this is to demonstrate that our most successful students are not so much and our least successful students are dreadful. If you look at the distribution of scores, you see exactly the same distances as any other test. The only difference is that the distribution has been manipulated to be 30 to 40 percent lower for everybody. This serves an enormously powerful purpose. If you establish a baseline this low, the subsequent growth over the next few years will indicate that your plans for elevating the outcomes were necessary. However, it must be recognized that the test developers control the scaled scores—indeed they have developed a draconian statistical formula that is elaborate, if indecipherable, to determine scaled scores. I would bet my house on the fact that over the next few years, scores will “improve”—not necessarily student learning, but scores. They must, because the State accepted millions and millions of dollars to increase student scores and increase graduation rates. If scores do not improve from this baseline, then those ‘powers that be’ will have a lot of explaining to do to justify having accepted those millions.
If you examine the distribution of the scores, the one thing that leaps off the page is the distance between children in high poverty and children in relative wealth. While all have been relegated to a point 30 to 40 percent lower than previously, the exact curve is absolutely connected to socioeconomic status—which has been historically true in such testing for more than a century.
The tragic part of this story is the collateral damage—the little children who worked so hard this year, who endured so many distressing hours of testing, who failed to reach proficiency, all because of the manipulation of the scaling. We will be talking with parents whose children scored level four last year, who now may have scored a level two. It does not mean much; it only means they are the unwitting part of a massive scheme to prove how these “high standards” are improving outcomes over time. It is time to pay attention to the man behind the curtain—he is no wizard, but he is wily!
LikeLike
I had already planned to post it next week, and will stick with my plan.
LikeLike
Linda, you are right there is no doubt the test scores will go up next year because now we have a sample of the tests in our hands. I can see it now…students will have copies of this year’s test plunked in front of them and will essentially memorize the test. Then NY State will produce what is basically the same test with new numbers etc plugged in…and there you have it, it’s teaching to the test at its finest.
LikeLike
Colleagues,
This is Dr. Thayer Snyder’s powerful post:
Commentary on Math & ELA Results
Dr. Teresa Thayer Snyder
Over the past several months school leaders have been receiving countless messages from the State Education Department preparing us for the dire outcomes associated with the most recent spate of State testing in grades 3-8 in Math and English Language Arts. As the date for the releases of the test scores approached, we received many notices of “talking points” to inform our communities about the outcomes, with explanations of new baselines and how these tests do not reflect the efforts of students and teachers this year. I have rejected these missives because they reek of the self-serving mentality the ‘powers that be’ have thrust upon our students and parents.
Our community is sophisticated enough to recognize a canard when it experiences one. These tests were intentionally designed to obtain precisely the outcomes that were rendered. The rationale behind this is to demonstrate that our most successful students are not so much and our least successful students are dreadful. If you look at the distribution of scores, you see exactly the same distances as any other test. The only difference is that the distribution has been manipulated to be 30 to 40 percent lower for everybody. This serves an enormously powerful purpose. If you establish a baseline this low, the subsequent growth over the next few years will indicate that your plans for elevating the outcomes were necessary. However, it must be recognized that the test developers control the scaled scores—indeed they have developed a draconian statistical formula that is elaborate, if indecipherable, to determine scaled scores. I would bet my house on the fact that over the next few years, scores will “improve”—not necessarily student learning, but scores. They must, because the State accepted millions and millions of dollars to increase student scores and increase graduation rates. If scores do not improve from this baseline, then those ‘powers that be’ will have a lot of explaining to do to justify having accepted those millions.
If you examine the distribution of the scores, the one thing that leaps off the page is the distance between children in high poverty and children in relative wealth. While all have been relegated to a point 30 to 40 percent lower than previously, the exact curve is absolutely connected to socioeconomic status—which has been historically true in such testing for more than a century.
The tragic part of this story is the collateral damage—the little children who worked so hard this year, who endured so many distressing hours of testing, who failed to reach proficiency, all because of the manipulation of the scaling. We will be talking with parents whose children scored level four last year, who now may have scored a level two. It does not mean much; it only means they are the unwitting part of a massive scheme to prove how these “high standards” are improving outcomes over time. It is time to pay attention to the man behind the curtain—he is no wizard, but he is wily!
By the way, if you want to know what curriculum experiences are being promoted for even our youngest learners by the ‘powers that be’, check out curriculum modules on http://www.engageny.org . How many of us truly believe that expecting first graders to understand and explain why Mesopotamia is the cradle of civilization is reasonable? How many of us truly even imagine that six year olds should be able to identify cuneiform and hieroglyphics or understand the importance of the code of Hammurabi? Check it out—then I suggest you let your legislators, and the Department of Education know what matters to you.
As we digest the information and prepare for the upcoming year, please rest assured that Voorheesville remains committed to challenging and cherishing our students.
LikeLike
Glad to see some of those in power finally finding the inner gumption to question the status quo. Welcome to the non-band wagon!
LikeLike