An earlier post linked to an article by Scott Waldman in the Albany Times-Union, in which Waldman pointed out that rankings accurately reflect the socioeconomic status of families in the schools, not the quality of the schools. He asked, what do those school rankings really demonstrate?
This letter came from a superintendent in the area of upstate New York about which Waldman wrote:
I am the superintendent of one of the high performing schools Waldman identified. He is, of course, absolutely accurate. The issue has always been poverty–urban, rural, and increasingly, suburban. It is easier and more expedient for politicians and naysayers in general, to attack schools–their costs, their teachers, their calendar, their curriculum–rather than address the root cause of the discrepancies–multi-generational systemic poverty. We have known about the impact of poverty on student achievement for hundreds of years. We have known how standardized test scores are skewed by zip code for years. Even the inventors of standardized testing (in the very early twentieth century) argued that they should be used judiciously because they are so sensitive to environment. I know an urban educator very well, who constantly states that it is not that his kids (grade five) can’t learn–indeed, they have already learned some skills about survival that are much more compelling than their ELA scores. The problem is that the things they have learned can not be reduced to a multiple choice test.

I think that for too long we who are in well resourced schools remained silent while schools overwhelmed by poverty were attacked. We were too willing to take the accolades, and by doing so we let the drumbeat of school failure get louder and louder. Thank you for speaking out. i hope that you will share your name and that you will get other colleagues to speak out as well.
This post shows how extensive poverty is in some districts and the relationship between poverty and graduation rates. http://roundtheinkwell.com/2013/07/02/poverty-and-graduation-rates-in-ny/
LikeLike
Wealthy, high performing districts have little to gain by speaking out against the reformers. The status quo works well for the top 10%, just like in real life.
LikeLike
Yes. And that is why it is important for supes like Teresa to speak out.
LikeLike
Actually, your point is well stated–but it is precisely WHY we who happen to run high performing districts need to address the issues. We can be smug– or we can be the front runners in addressing the theater of the absurd that swamps us all. I, and my colleagues in high performing districts, could sit back and accept our “just desserts” or we can address the profound inequities that the data reveals–the clarion call that the “emperor has no clothes” should come from those of us who might have been the tailors!!
LikeLike
… and the attack on well-resourced schools is happening now too… I have a friend who works in a solidly middle-upper income area in Illinois… their “cut score” for high-stakes testing has changed… so now instead of 90 percent of students in passing range, it looks like only 70 percent of the students are “passing.” And can you guess what textbook/software companies have swooped in to provide the medicine for the cure? I bet you can!
LikeLike
Thank you Teresa! I have heard wonderful things about you and your courage. I am grateful for this opportunity to acknowledge your good work and your caring beyond the boundaries of your district.
LikeLike
I am happy to provide my name. I am Teresa Thayer Snyder and I am superintendent of the Voorheesville Central School District, just west of Albany, NY. While I have always commented on the discrepancies in performance based on socioeconomic status, I also comment on the fact that even students who are scoring well and graduating in large percentages are not necessarily reaping academic benefit from the horrific amount of time expended on the latest standardized testing frenzy. It is not about raising standards, it is about compliance with highly suspect measures and the inappropriate applications of the data they produce. There are other measures that are of concern which actually may be indications of equality across socioeconomic status–I believe measures of anxiety associated with the recently administered “high stakes” tests would be a common starting point! Once, when I was a principal, I noted the spike in visits to the nurse’s office during “testing season”–a ton of research exists on the negative impact of stress on performance.
LikeLike
Teresa,
Have you read Noel Wilson’s As shown by Noel Wilson in “Educational Standards and the Problem of Error” found at: http://epaa.asu.edu/ojs/article/view/577/700 ? If so, what are you’re thoughts of the invalidity of the whole educational standards and standardized testing regimes?
If you believe that these are wrong policy what have you done to prevent the harm to the students in the VCSD? Have you instructed principals to have the teachers “not teach to the test”? Have you vociferously challenged the powers that put these schemes in to place? Or are you just learning now about these insanities? I challenge you to challenge the status quo.
Thanks and here’s hoping that you are/will be fighting the standardized regime with full vigor, especially after reading and understanding why it is so pernicious.
Duane
LikeLike
In response to Mr. Swacker’s questions: I have read the Wilson article and I have spent the better part of my career challenging standards. I also have, in my many years of teacher contact, urged them to do anything but teach to the test–these tests are instructionally insensitive–Jim Popham, professor emeritus from Stanford and an expert psychometrician, has written about that insensitivity in many venues. Students are instructionally sensitive, however, and I urge the teachers who have worked with me to challenge them, and not to worry about the standards. The current scheme has education turned upside down and it is up to us to right this, which we must do with reasoned, researched consistency.
LikeLike
Thanks for the reply and the work you are doing to combat the insanity. And you must be a true “BadAss Teacher:” Coming from one who chose not to join the BAT, but who is a Mr. Teachbad certified “difficult teacher”-ha ha!
LikeLike
And, Teresa, as we here all darn well know is that the “standardized” tests are not–they are neither valid nor reliable.
The scoring has been and undoubtedly continues to be juked.Not only are our kids NOT learning, what they ARE being taught (as pupils of teach-to-the-test-ers) is…how to take tests. More specifically, how to take NOT standardized tests run by for-profit agencies (the monopolizing Pear$on). However, at the end of their education, our students will have “21st Century life skills.” (No critical thinking, no ability to question authority or to challenge what is wrong, etc.)
LikeLike
Well, Theresa, if you are in charge, why do you let it happen? Don’t we have a moral obligation to disobey immoral laws (as Martin Luther King says in “Letter From Birmingham Jail.”
LikeLike
Harlan if she does she loses her license. And then we lose her voice. I am sure she does what she can, and I am glad we have her.
LikeLike
Harlan, the rules and politics in public education are considerably different than in private settings even if in both wrong is wrong.
LikeLike
My goodness, economist, I hardly think that BC Calc in the senior year is holding students back! We will disagree. You believe in tracking to benefit the elite. I do not. We are not alone in our disagreement.
LikeLike
I believe in tracking to benefit students.
When you see high school students taking Ph.D. Classes in mathmatics, BC calculus in the senior year does seem to me to be holding students back.
LikeLike
I suppose one possibility in your system would be for students to skip grades. Is that commonly done?
LikeLike
Carol,
To play devil’s advocate for a moment (and I’m with you 100% – I’m one of those teachers in a poverty stricken school and came from another that was closed).
Schools are often touted as the great equalizer (I think that may be how we got here as education being the “Civil Rights Issue” of our time. How is this argument reframed if we postulate that education can’t overcome poverty – what is the path then to solving poverty if education is to play that role if it plays it at all.
LikeLike
I can’t speak for Carol, but I’d suggest that public education should develop the citizenship skills that could eventually translate into what used to be called industrial democracy, whereby working people have more power in the labor markets and some say on the job, and students would arrive in school with their basic needs having been met.
Of course, with so much disenfranchisement – which is what mayoral control of the schools represents – and union busting, industrial/economic democracy is a taboo discussion topic among so-called reformers. For them, the empty mantra of “college and career readiness” trumps all. In other words, shut up, take on your student loan debt, and get to work.
LikeLike
To rebut in the language of what i’m sure many would say to that – if we ensure that every individuals needs are met by the government – as we’d need to sink the resources into that to make that a reality – would we not be moving into a socialist state whereby the government controls our resources and thus our choices instead of the individual?
Put another way without the socialist tag – how the heck could we pay for it and create an incentivized system for improvement, if everyone was absolutely 100% sure they’d have a somewhat comfy life because of government support.
LikeLike
Edit: I meant this for the comment on providing for people.
LikeLike
And where exactly did I say that children’s needs should be met by the government? In the future, you should try to keep your assumptions from blinding you to the text in front of your eyes.
Economic/industrial democracy has the potential to raise living standards, and educational outcomes, organically throughout the economy, by shifting the balance of power – currently maintained by the heavy hand of the state favoring Capital, especially wealth-extracting finance capital – and national income towards Labor.
LikeLike
If I may condense, what I see you saying is that all students should be educated in school to vote the labor ticket. Is that correct?
LikeLike
Poverty is the issue. However, no one is saying that it is the obstacle to a great education and getting out of poverty. I would suggest that schools with a high FARM (Free and Reduced Meals) population should receive a larger amount of resources than schools with low amounts. Our funding system is not equitable but fair. And as I tell my students “Fair is not always equal”. By providing wrap around services with medical, dental, nutritional, psychiatric and parental care bestowed upon those families and children in poverty I believe you will see a turn around. Also, by implementing early childhood education to those communities that are struggling with poverty we will see substantial improvements. So, although zip code does play a part in how well a student performs it doesn’t have to be the downfall for that student.
LikeLike
Here is the way I see it, M. Take a look at how well low SES kids do in low needs schools–85% four year graduation rate. So what can we learn and what can we do?
Let’s use my school as an example. About 15-16% of my students receive free or reduced priced lunch. Two did not graduate this year…they will next year. Of the rest, 100% graduated with a Regents diploma.
Why? Because I have 3 social workers, 2 psychologists, 2 nurses, class size average of about 26, support classes. Much of those resources directly benefit that 16%.We have a rich program in the arts that makes kids want to come to school.
In addition, we have no tracking in Gr 9 and 10 and minimal in 11 and 12. All of our kids take IB English. That means that kids who have little (and little hope) are sitting with, learning with, talking with kids who are living in homes that benefited from education.
We are not overwhelmed by high needs kids. I have the time to walk kids to extra help, to hunt down kids who do not go to school, to do home visits.
That is what low needs schools can do. So if I have 3 social workers, how many should your school have? If my class size is 26, what should your class size be? These are the questions we should ask.
The second big question, is why do we allow high poverty, apartheid schools to exist? Why have we abandoned integrated neighborhoods and schools? In suburban communities in Virginia, communities must have affordable housing.
But politicians do not want to deal with this big issues. When I brought up integration to Arne Duncan, his response was, “that’s hard”. Yes it is. The work is hard. And what angers me is that every day and every dollar wasted on faux reform, brings us that further away from the work we must do.
LikeLike
In my local high school some in tenth grade take pre calculas, others take algebra 1. Do all your tenth grade students take the same math class? If so, which one is it?
For background, my children have all attended the “rich” high school in town where only 25% of the students are on free or subsidized lunch. The other high school has 40%.
LikeLike
For teaching economist….all take algebra in 8, geometry in 9, advanced algebra/trig in 10. Kids who want more, take an every other day advanced topics course along with advanced algebra.
LikeLike
What paths after 10? In our local high school strong students take AP calculas in 11, the strongest go to the university in 11 to take the 10 hour science engeneering calculus sequence, and the strongest of those will end up in graduate math classes by the time they graduate from high school (only one every three years or so).
LikeLike
11th grade–IB math methods or math studies
12th grade–AB calc, BC calc and or AP stats
When you worry about “the strongest”, you leave everyone else behind.
LikeLike
Well that is one way to narrow the gap.
Don’t you find that you get some frustrated students, especially with all of the alternative sources of math instruction on the web?
LikeLike
Here is an example of the sort of student I think we will be seeing more of as schools become only one source students can turn to in order to learn. This was a post on the website math stack exchange
I am still 15 years old, but I am very interested in pure math. I have been teaching myself though books, from the internet and from others for the past year or so. I haven’t mastered all the topics that are covered in university, just the ones that happen to interest me (elements of differential and integral calculus, complex analysis, etc. You can see what I am interested in by looking at the questions I’ve asked and answered).
Now, a few months ago, back in school, I asked my math teacher for help on a differential calculus question whose solution I did not understand. I was told by this teacher that I should not be doing calculus and I should wait until I learn it in school. Other math teachers either did not understand what I was asking or shared the same view as my math teacher. For awhile this had distressed me very much, because some of my own math teachers were telling me to stop learning math and to wait three or four years to continue! Should I stop learning math by myself? I decided that I would keep going, because this is a hobby and interest of mine and I didn’t think teachers should have the right to stop me from learning.
Most of the comments on this post disparaged the teachers and urged the student to seek out college or university classes and instructors.
LikeLike
carolcorbettburris: your candor and integrity are much appreciated.
I wish there were more administrators like you.
LikeLike
No, I do not find they are bored. They understand that learning in a community has value. Our teachers are very good at going deeper when needed. It is a public school with good values.
LikeLike
Well, as I said, holding strong students back is probably the simplest way to close the gap.
I suppose there are private schools for the strong students who come from families that can afford them.
LikeLike
Now that is definitely a commendable program you are running there. That everyone takes IB English is really something. I assume, however, that not all students pursue an IB endorsed degree which would require high level work in the other five areas of the IB curriculum as well as sitting external exams in both Junior year and Senior year. Is that correct?
Because you are a high resource and low needs district you actually can do what is right by all the children, as Diane argues should be done across the country.
I’ll bet you don’t attack the wealthy the way she does, however, since the resources of your district depend on the wealth creating activities of your residents. You’d be biting the hand that not only feeds you, but permits you to extend your hand to all students in your district, including the poor.
LikeLike
I have to disagree that this school does right by all students. This principle seems to be very concerned that some students will be left behind, so all students have to progress at the pace of the average, orpernaps low average student, at least until 11th grade.
One student from my local high school was offered admission to Stanford’s graduate program in mathamatics straight out of high school. That would have been an impossibility in the one size fits all high school discribed here. She would certainly not been well searched at this high school.
LikeLike
I’d say that the parents of exceptional students will tend to clamor for exclusivity (tracking, etc.) because they perceive that to be best for their children, and the parents of below average (or special education) students will tend to clamor for inclusion because they perceive THAT to be best for THEIR children. They may both be right, and if they are, then there is no way for a school system to win, and many questions that critics can pose with smug smirks.
LikeLike
The best public policy is made by basing decisions on the world that exists rather than the world that we wish existed.
If Carol Corbett Burns is correct and we can not have a school or school system that provides the best education for all, we need to think about which students we will design a school system around, how large an impact that will have on the students who will not be given the best education they could have and how much we care about the losses those students will suffer.
LikeLike
Oh, here’s a fellow advocates leaving some children behind! Certainly not his OWN, though… (smug smirk)
LikeLike
Carol was advocating holding students back and saying that allowing some ahead would hurt others. She is the one claiming there is no way to allow every student the education they can take advantage of.
LikeLike
Harlan,you are correct. Not all students do the whole IB diploma. About 50% do.that is there choice. 34%of the entire graduating class last year and this year earned it.
The parents in my district are hardly billionaires, some would be described as upper middle class. We work to serve all students well regardless of family wealth. Kids are kids and we care about all.
LikeLike
Argh, or perhaps, not orpernaps.
In any case there have been several enlightening comments lately. Carolcorbettburris’ comment that paying attention to the most able students result in leaving everyone else behind suggests that there is no possibility of having a better education for all, we have to choose who gets an education that fulfills all their potential.
I would look forward to a discussion of this on the blog, but I do not expect it.
LikeLike
Excellent question, M.
LikeLike
In my perfect world I’d spend money on reducing the stress in the home. Offer what the rich have. Parent goes to work – make sure there is plenty of nutritious food and a chef to cook. Have proper supervision and make sure a child is getting their physical and emotional needs met. How about a nanny in every home. Make sure children have proper medical and dental care. Make sure they can live in a home, which they will be allowed to stay in with proper heating or cooling systems. Give children support to guide them on how to learn and the tools to make this an option. Even if the parent(s) are in low paying jobs they should be able to support their child’s environment in this manner. Some countries actually give mothers a stipend to stay home and care for their children. Support is my number one starting point and it should start very, very early before a child enters school.
LikeLike
It actually should start pre-natally.
LikeLike
On the mark, but what we need is for superintendents to join together, stand up publicly with their staff, and fight. Don’t just send letters to Diane Ravitch – send them to every newspaper in NYS.
LikeLike
Don’t just send letters, refuse to take part in the testing insanity.
Oh, I know they may lose their certification, too bad, then go back into the classroom.
LikeLike
Just ensuring free college education for all would go a long way. How many kids don’t bother getting serious about high school because they know they won’t be able to afford college without taking on a massive loan — which may be too daunting a risk if no one else in their family has attended college?
Can’t do that, though — it would keep too many people out of indentured servitude to the banks!
LikeLike
From my perspective the most disturbing thing I hear about is students dismissing poor performance in high school because they can always go to the university where I teach. Of course we are fairly cheap at $9,000 or so a year tuition for in state students, so money may be less of a concern.
LikeLike
$9,000 a year is quite prohibitive for many a student in this rural poverty district. When your family has nothing that amount is an insurpassable barrier.
LikeLike
I agree that $9,000 is a lot of money, but it is just the sticker price. It can be reduced by a variety of scholarships based on high school GPA and/or SAT ACT scores. The cost of a degree can be reduced even further by taking the first couple of years at a community college and transferring the credit hours.
LikeLike
Is your University therefore presitigious? Or not?
LikeLike
I don’t know if it is or not. I have had students who have turned down MIT, Dartmouth, and the University of Chicago among other schools.. All that is required for admission is a 980 on the SAT OR a C average for a set of academic classes OR graduating in the top third of the class. Does that make it prestigious?
Apologies for the double post the first ended up in the wrong place.
LikeLike
“The problem is that the things they have learned can not be reduced to a multiple choice test.”
This is what I found to be true when I taught students in an poor urban school district. My students were labeled (some I believe incorrectly) as being educable mentally handicapped strictly based on an IQ test. To me they were wise beyond their age especially when it came to being able to survive in the housing projects they lived in.
LikeLike
Ron I agree. My mother lived in poverty and spent 5 years in an orphanage, along with her siblings in the ’30s. Her brothers all enlisted in the army or navy and two of the three went to college on the GI bill. One became a college professor, the other received an economics degree from Columbia university and went on to be very successful. The other was a successful glazier.
LikeLike
And he is right of course. There is nothing like an urban kid for survival skills. When schools were on holidays in Atlanta, they would find food by going to the soup lines standing right there with the homeless people. They rode the buses at 8 years old, little brothers and sisters in tow. Even if they couldn’t read or add they could handle money. Urban kids are smart and resourceful. Too bad they can’t measure those skills on standardized tests.
LikeLike
It is helpful to read these Albany news articles and remind ourselves of all the intervening variables. This superintendent’s letter also is helpful. So much of the news media (I am thinking in particular of Boston area — WBUR, etc) just keep repeating the same old headlines of “teachers fail ” “teaching reading is failing” and these are old headlines (as the superintendent points out– poverty is always with us). Whenever I see these headlines (e.g., “Teachers are pricey” being picked up by UPI) I write an email or call. As Diane pointed out we need more of the students in Rhode Island getting press releases and headlines.. As Bill Cosby says “you young people have to do the work” but we need to support them with the decades of research that is available if you go beyond the newspaper headlines. If any one is looking I will help dig it out. We know for decades the real estate people have been using their “data driven” mode to sell homes in wealthy districts. Those of us who have experienced decades of this have to speak out. There are good people working on the issues like Diane, Marilyn Cochran Smith, Linda Darling Hammond etc but the excellent studies do not hit the daily press that wears us down with this repeated “failing” mantra . I know there are good faculty members working on state committees in Massachusetts and they don’t feel that anyone knows the truth about the successful work they have done because the political agenda is driving this constant barrage (WBUR puts out on their elitist “cognoscenti” reading teachers fail” for example. Any headline with the word “fail” will draw the readers who are mostly uninformed but would like to shut down anything that brings “gubmint ” to mind. My former boss did one of the early dissertations at BU that showed the facts about how the funding formula continues to leave schools in poorer districts far from the opportunities that are necessary in life and J. Hassard has pointed this out in his reviews of current “data driven” states that are looking to fire teachers because of low test scores. A concerted effort is needed to bring forth the efforts of so many educators that have devoted lifetimes to the major educational goals. Federal funds were wisely used in health/special education, teacher preparation in Massachusetts and I attribute that to the success experienced. The purpose of Knowledge Production and Utilization with a strong dissemination and diffusion arms/legs has seemed to devolve into individual competitions among states/districts etc and this has left a huge gap where slick marketing can move in with 5 colored brochures to “sell”. This was not the goal of KPU research centers (J. Hassard describes it better in his Art of Teaching Science) ….. We need a complementarity between “top down” and “bottom up” to contribute to the endeavors of our educational goals. In these economic struggles today there is a quick “bandaid” fix that seems to be the default for politicians and a major “drum roll ” of nihilistic thinking and profit minded marketing strategists that have united to defeat major goals and purposes of education in the traditions that we have known and that we believe are essential for the future.
LikeLike
In reference to the issue or urban poverty, I can quote one superintendent from a major urban district who implemented computer assisted instruction developed by Pat Suppes. This was a well researched program of math and it was supplementary to the regular classroom (for 10 minutes a day, with teachers in Chapter I /Title I classrooms.. Chapte I is designed to be supplementary). The superintendent I am quoting said to us “when I put in Pat Suppes math program I was getting test scores equivalent to the Catholic schools in my district.” This is testimony and you don’t have to believe but the superintendent was doing something to help the students by choosing a well researched program of CAI. All of the individuals who then bought a desk top computer (or TRASH 80 — sorry for the pejorative name here) thought they could invent their own math curriculum and it would be effective . That is like saying to a parent your child needs orange juice and the parent gives them orange-colored kool-aid. The evaluative research done with Pat Suppes BEFORE the program was disseminated widely was front-ended (instead of the companies now expecting the school districts to pay for field testing.) Also, Pat Suppes CAI math program got effective results in MATH in year one but it took two or more years to raise the reading scores. I can cite 5 districts who implemented the CAI program and it was mediated by the experience of the teachers so that results varied across the districts. In one district the teachers said “we pretend the computers are broken” and they did not get the results on math scores because program fidelity was absent. The urban superintendent I am quoting who got effective math scores also instituted a major staff development program across the district so it is not possible to draw cause-effect conclusions at the 95% level. There were strong indications that there was a teacher effect in all of the scores that I saw by districts. And, the standardized test used to measure the efficacy of the CAI was not developed by the CAI marketing/distributing firm but it was well known with a long tradition of use in schools (with city norms, etc) I’m “on your side” but sometimes we need to look at details and not over generalize.
LikeLike
“. . . I put in Pat Suppes math program I was getting test scores equivalent to the Catholic schools in my district.”
Any public school superintendent worried about “getting test scores equivalent to the Catholic schools in MY district” ought to be fired. The bogus “measure” that is standardized test scores is part and parcel of the main problem with public education today, that of “standardization” and the sorting, separating and ranking of students, teachers, schools and districts. Scores are the cart before the horse-the teaching and learning process.
Jean, I invite you to read just why the whole educational standards and standardized testing, grading of students regime is completely invalid in Wilson’s “Educational Standards and the Problem of Error” found at: http://epaa.asu.edu/ojs/article/view/577/700 . Why even a standardized test that supposedly “was well known with a long tradition of use in schools” is still a BAD and INVALID mechanism for the teaching and learning process.
LikeLike
Here’s a quick summary and some of my comments to whet your appetite:
A brief outline of Wilson’s “Educational Standards and the Problem of Error” and some comments of mine. (updated 6/24/13 per Wilson email)
1. A quality cannot be quantified. Quantity is a sub-category of quality. It is illogical to judge/assess a whole category by only a part (sub-category) of the whole. The assessment is, by definition, lacking in the sense that “assessments are always of multidimensional qualities. To quantify them as one dimensional quantities (numbers or grades) is to perpetuate a fundamental logical error” (per Wilson). The teaching and learning process falls in the logical realm of aesthetics/qualities of human interactions. In attempting to quantify educational standards and standardized testing we are lacking much information about said interactions.
2. A major epistemological mistake is that we attach, with great importance, the “score” of the student, not only onto the student but also, by extension, the teacher, school and district. Any description of a testing event is only a description of an interaction, that of the student and the testing device at a given time and place. The only correct logical thing that we can attempt to do is to describe that interaction (how accurately or not is a whole other story). That description cannot, by logical thought, be “assigned/attached” to the student as it cannot be a description of the student but the interaction. And this error is probably one of the most egregious “errors” that occur with standardized testing (and even the “grading” of students by a teacher).
3. Wilson identifies four “frames of reference” each with distinct assumptions (epistemological basis) about the assessment process from which the “assessor” views the interactions of the teaching and learning process: the Judge (think college professor who “knows” the students capabilities and grades them accordingly), the General Frame-think standardized testing that claims to have a “scientific” basis, the Specific Frame-think of learning by objective like computer based learning, getting a correct answer before moving on to the next screen, and the Responsive Frame-think of an apprenticeship in a trade or a medical residency program where the learner interacts with the “teacher” with constant feedback. Each category has its own sources of error and more error in the process is caused when the assessor confuses and conflates the categories.
4. Wilson elucidates the notion of “error”: “Error is predicated on a notion of perfection; to allocate error is to imply what is without error; to know error it is necessary to determine what is true. And what is true is determined by what we define as true, theoretically by the assumptions of our epistemology, practically by the events and non-events, the discourses and silences, the world of surfaces and their interactions and interpretations; in short, the practices that permeate the field. . . Error is the uncertainty dimension of the statement; error is the band within which chaos reigns, in which anything can happen. Error comprises all of those eventful circumstances which make the assessment statement less than perfectly precise, the measure less than perfectly accurate, the rank order less than perfectly stable, the standard and its measurement less than absolute, and the communication of its truth less than impeccable.”
In other word all the errors involved in the process render any conclusions invalid.
5. The test makers/psychometricians, through all sorts of mathematical machinations attempt to “prove” that these tests (based on standards) are valid-errorless or supposedly at least with minimal error [they aren’t]. Wilson turns the concept of validity on its head and focuses on just how invalid the machinations and the test and results are. He is an advocate for the test taker not the test maker. In doing so he identifies thirteen sources of “error”, any one of which renders the test making/giving/disseminating of results invalid. As a basic logical premise is that once something is shown to be invalid it is just that, invalid, and no amount of “fudging” by the psychometricians/test makers can alleviate that invalidity.
6. Having shown the invalidity, and therefore the unreliability, of the whole process Wilson concludes, rightly so, that any result/information gleaned from the process is “vain and illusory”. In other words start with an invalidity, end with an invalidity (except by sheer chance every once in a while, like a blind and anosmic squirrel who finds the occasional acorn, a result may be “true”) or to put in more mundane terms shit-in shit out.
7. And so what does this all mean? I’ll let Wilson have the second to last word: “So what does a test measure in our world? It measures what the person with the power to pay for the test says it measures. And the person who sets the test will name the test what the person who pays for the test wants the test to be named.”
In other words it measures “something” which supposedly is specified by the test maker but the whole process is so error ridden that any conclusions drawn are invalid. The test supposedly measures “’something’ and we can specify some of the ‘errors’ in that ‘something’ but still don’t know [precisely] what the ‘something’ is.” The whole process harms many students as the social rewards for some are not available to others who “don’t make the grade (sic)” Should American public education have the function of sorting and separating students so that some may receive greater benefits than others, especially considering that the sorting and separating devices, educational standards and standardized testing, are so flawed not only in concept but in execution?
My answer is NO!!!!!
One final note with Wilson channeling Foucault and his concept of subjectivization:
“So the mark [grade/test score] becomes part of the story about yourself and with sufficient repetitions becomes true: true because those who know, those in authority, say it is true; true because the society in which you live legitimates this authority; true because your cultural habitus makes it difficult for you to perceive, conceive and integrate those aspects of your experience that contradict the story; true because in acting out your story, which now includes the mark and its meaning, the social truth that created it is confirmed; true because if your mark is high you are consistently rewarded, so that your voice becomes a voice of authority in the power-knowledge discourses that reproduce the structure that helped to produce you; true because if your mark is low your voice becomes muted and confirms your lower position in the social hierarchy; true finally because that success or failure confirms that mark that implicitly predicted the now self evident consequences. And so the circle is complete.”
In other words students “internalize” what those “marks” (grades/test scores) mean, and since the vast majority of the students have not developed the mental skills to counteract what the “authorities” say, they accept as “natural and normal” that “story/description” of them. Although paradoxical in a sense, the “I’m an “A” student” is almost as harmful as “I’m an ‘F’ student” in hindering students becoming independent, critical and free thinkers. And having independent, critical and free thinkers is a threat to the current socio-economic structure of society.
LikeLike
I’m curious.
The commenter “M” questions how education can “solve” poverty.
It can’t. It can help, because those who are better educated tend to have more knowledge, better skills, more developed insights into parenting, and better-paying jobs.
Although, to be sure, more education does not automatically translate into an enhanced sense of morality and ethics. The recent mortgage and financial crises –– and the ongoing attempts by corporate “elites” to fatten their wallets from taxpayer subsidies –– make that perfectly clear.
But government has a role – a significant one – to play as well. The critics of government like to throw out the term “socialism.” But the Constitution was written to “create a more perfect union,” and to “establish justice,” and to “promote the general welfare.” Alleviating the pernicious effects of poverty surely falls within the context of those founding principles. So does solidifying the social contract (see Ron Poirier’s comment on college education above).
And while public schooling can help to equip individuals with knowledge and skills, it should also imbue them with the core values of democracy; it should help to develop and maintain democratic character.
So I’m curious. What does “M” see as the purpose of public education? Or, as John Goodlad (or John Dewey) might ask: In a democratic society, what are public schools for?
LikeLike
I thought the purpose of education was whatever Bill Gates SAYS it is, at that moment.
LikeLike
The answer to your question lies in each state constitution.
How many here know what your state constitution says in the public school authorizing section/article? (Hint, I doubt any one of them says anything about “career and college” ready.)
LikeLike
As I stated – I was simply providing the counter-point to sharpen my rhetoric because I can imagine someone saying what I said – what a legitimate response would be I’m not so sure of.
I can counter with what I know about education – that my job is to build independent life long learners who are ready to participate as citizens in a democratic society and in my particular field – that I’m supposed to equip them with the tools to question and compare different forms of information and come to independent conclusions based on multiple points of view while instilling a love of literature (which translates these days into literacy – a poor substitute).
As for what I would say in response to Carol – I truly appreciate her response and I legitimately have spent about 2 hours processing what it would take to make that vision work of having sufficient resources for schools to counteract these effects that get in our student’s way.
The ultimate question in developing her system aside from cost then – is then how do we legitimately separate and de-segregate our system which is far different from the systems we knew over 50 years ago where you’d have 2 schools for people of different color. In this case, you may have entire counties or districts that are segregated from others via haves and have-nots.
So you’d literally have to identify “needy” vs. “not high need” students, and then, choose a decent percent (say at least 10%) of a low need schools population that are also not “high needs” or “at risk” students, and send several schools worth of them to one school to bring down that school’s percentage of high needs students. Simultaneously, you’d have to take say this one school of high needs students and force the non-high need schools to take them (assuming they have a choice). And all of this doesn’t take into account geography and all of the issues with basing school funding on property tax levies and the “fairness” to that community of people that don’t pay taxes to that community, then having that school devote a significant chunk of their resources to solving their issues. Simultaneously, you’d need to convince those parents that it’s only fair that their children get sent farther away to a “worse” school because it’s fair to society and creates an equitable educational system that responds to the needs of high needs students.
I want to come up with an answer for how we could make that a reality – I don’t have one. I now see why they favor “choice” as the way to try to solve this problem so children aren’t “stuck” even though there may be VERY few pockets of non-high needs schools to choose from. It creates an illusory solution to the problem that makes everyone “happy” but doesn’t solve the problems of having several pockets of very high needs students all in the same zip code.
I simply, other than my philosophical basis for how I teach, do not have an answer for how we could really make that work.
LikeLike
Problem is, M, choice as it happens now in NYC makes it worse. I just did research on it for my new book. You can do controlled choice that reserves seats, you can do controlled magnets, you can do suburban-urban programs, and you can add affordable housing to communities… you do it scattered and in the small.
LikeLike
I don’t know if it is or not. I have had students who have turned down MIT, Dartmouth, and the University of Chicago among other schools.. All that is required for admission is a 980 on the SAT OR a C average for a set of academic classes OR graduating in the top third of the class. Does that make it prestigious?
LikeLike
Since the SAT doesn’t measure much more than family income, why is a 980 a problem?
LikeLike
First, I disagree that the SAT doesn’t measure much more than family income, but that is beside the point.
This posted in the wrong place, and was in response to a question about the prestige of the university where I teach. The most prestigious universities typically do not admit students with 490 490 SAT scores or a C average in high school.
LikeLike
democracy and TE:
“Since the SAT doesn’t measure much more than family income, why is a 980 a problem?” (from democracy) and “First, I disagree that the SAT doesn’t measure much more than family income, but that is beside the point.” (from TE)
First, the SAT is not designed to “measure” family income. So, no it doesn’t. Now there is a correlation between standardized test scores and family income? Yes, but correlation does not equal “measure”.
It may seem to be a semantic game I am playing but I prefer as much precision as possible in our (those of us who are denying the validity of educational standards and standardized testing) discussions of standardized testing and its pernicious effects and, in general, any discussions of educational policy and practices.
The SAT has all the inherent errors that render it invalid as all standardized tests as shown by Wilson (see above postings for summary) and any results/conclusions drawn from a student’s score are “vain and illusory” and should best be digested with a block of salt.
LikeLike
Does the SAT really measure not much more than family income? I’ve heard this, and agree with it to a point, but I also scored pretty high on mine — 1340, if I remember correctly, and this was in 1990. My father taught at a Catholic school and my mother worked part-time at CVS. We weren’t exactly poor, but we certainly weren’t wealthy. Only my father had attended college, and he was part of the first generation in his family to have done so.
My family members tend to score well on standardized tests as well; most of the last generation became teachers (with the exception of one engineer and one scientist).
I believe family income has significant impact, but I don’t think the test measures that alone.
Something I sometimes bring up to critics is that MY kids will do fine in the new regime — they test well, and if high test scores are rewarded then they will reap the rewards. I protest our testing mania mostly because I see how it will hurt OTHER people’s kids.
LikeLike
Ron,
The SAT reflects family income but many poor students do well and some rich kids do poorly. Find the chart on the SAT annual report where they show how test scores correlate with income. Highest income has highest scores. Lowest income has lowest scores. As income goes up, so do scores.
LikeLike
Diane, I agree that there seems to be a correlation between family income and SAT score. What I’m asking is if that is really ALL that it measures. Given the correlation, what does it mean when a child from a low income family scores very high on the SAT, or when a child from a high income family scores very low? Are there trends in the outliers?
Why did I score quite well on the SAT when I took it — why do I tend to score well on standardized tests of all types — when my family didn’t really have much in the way of financial wealth? It’s a fair question.
Would it be useful to take SAT scores, for example, and use some method to take family income into account?
LikeLike
Does anyone know how closely teacher assigned grades corrolated with income?
LikeLike
@ Teaching Economist (and others):
There are certain myths in education that just don’t seem to die easily.
One, of course, is the claim that public schools are “in crisis” and need a healthy dose of “reform.” The Sandia Report (1993), the 2009 PISA results, and the most recent NAEP scores prove that claim untrue.
A second myth is that more testing, and charter schools (and vouchers), and merit pay will fix the (nonexistent) “crisis.”
A third is that there is a STEM shortage. Patently untrue.
And a fourth is that the SAT and ACT actually predict college success. They don’t.
The ACT is only marginally better than the SAT at predicting college success. What both measure best is family income, and colleges know it.
Matthew Quirk reported on this in “The Best Class Money Can Buy:”
“The ACT and the College Board don’t just sell hundreds of thousands of student profiles to schools; they also offer software and consulting services that can be used to set crude wealth and test-score cutoffs, to target or eliminate students before they apply…That students are rejected on the basis of income is one of the most closely held secrets in admissions; enrollment managers say the practice is far more prevalent than most schools let on.”
http://www.theatlantic.com/magazine/archive/2005/11/the-best-class-money-can-buy/4307/2/
And the net result is this: “More and more, schools are chasing the small number of students who have the money or the test scores that help an institution get ahead. As those students command higher and higher tuition discounts, they leave a smaller and smaller proportion of the financial-aid budget for poor students, who are increasingly at risk of being left out of higher education.”
As to the ACT, like the SAT, it too is far more hype than a genuine educational tool with merit.
The authors of a study in Ohio found the ACT has minimal predictive power. For example, the ACT composite score predicts about 5 percent of the variance in freshman-year Grade Point Average at Akron University, 10 percent at Bowling Green, 13 percent at Cincinnati, 8 percent at Kent State, 12 percent at Miami of Ohio, 9 percent at Ohio University, 15 percent at Ohio State, 13 percent at Toledo, and 17 percent for all others. Hardly anything to get all excited about.
Here is what the authors say about the ACT in their concluding remarks:
“…why, in the competitive college admissions market, admission officers have not already discovered the shortcomings of the ACT composite score and reduced the weight they put on the Reading and Science components. The answer is not clear. Personal conversations suggest that most admission officers are simply unaware of the difference in predictive validity across the tests. They have trusted ACT Inc. to design a valid exam and never took the time (or had the resources) to analyze the predictive power of its various components. An alternative explanation is that schools have a strong incentive – perhaps due to highly publicized external rankings such as those compiled by U.S. News & World Report, which incorporate students’ entrance exam scores – to admit students with a high ACT composite score, even if this score turns out to be unhelpful.”
So, the explanation has two general possibilities, neither one very good. The first is that college admissions staff (and top college administrators) are ignorant of the severe “shortcomings” of the ACT and SAT. If that is the case, why in the world do these people have jobs in academia? They may as well be doctors who still practice bloodletting.
The second possibility – the real answer – is that they engage in this nonsense (and it really is nonsense) willfully. Because it’s all about “image”…and money. For example, Matthew Quirk noted that former VCU president Eugene Trani used to carry “a laminated card in his pocket to remind him of the school’s strategic goal of making it to the next tier. For every year the school stays in the higher tier he will receive a $25,000 bonus…” He was – and is – surely NOT the exception. In a very real (and nefarious) sense, this is a continuation of who gets voted “best dressed” in high school. It’s the real-world equivalent of giving tax cuts to the already wealthy to “stimulate” the economy. Perhaps the fictional Forrest Gump described it best: “Stupid is as stupid does.” But stupid carries some very real consequences.
LikeLike
You will be pleased to observe that my university does not require SAT or ACT scores, any in state student with a C average over a small set of academic classes is automatically admitted.
Surely any student capable of earning a 2.0 is capable of graduating from an AAU university.
LikeLike
I emailed this particular blog entry to one of my district’s administrators with the following note.
“I read this and thought ‘So true.’ Then I thought of [ ] this summer [preschool student in the summer school session I taught]. Yes, he’s definitely ‘already learned some skills about survival’—but how will he fare in the educational system as it is today? Will he thrive in it, or will he thrive in spite of it?
This is the one-liner I received in reply.
“Thank you for sharing. It is unfortunate that accountability is the reality of today.”
Not what I was hoping or expecting to receive. Are we now zombies?
LikeLike
Administrators gotta administrate.
I know several administrators who, in private, are dead-set against corporate education reform, but back it to the hilt in public (or at least sit there like neutered cats and say nothing on those occasions when the reforms are attacked in public). It’s understandable — their jobs depend on their at least paying lip service to the church of corporate education reform.
LikeLike