It is indisputable that standardized tests have a disparate impact on members of minority groups. Asian students perform exceptionally well on math tests. White and Asian students have higher scores than black and Hispanic students.
The same disparate impact is found on teacher tests as on student tests. Occasionally, an African-American teacher will sue, claiming test bias. They usually win. New York State’s teacher licensing exam attempted to be “more rigorous,” but making them harder to pass cemented the gap between the pass rates of different racial groups.
“On a common licensing exam called Praxis Core, a new test given in 31 states or jurisdictions that was created to be more rigorous than its predecessor, 55 percent of white candidates taking the test since October 2013 passed the math portion on their first try, according to the preliminary data from the Educational Testing Service, which designed the exam. The passing rate for first-time African-American test takers was 21.5 percent, and for Hispanic test takers, 35 percent. A similar gap was seen on the reading and writing portions.
“In New York, which now has four separate licensing tests that candidates must pass, an analysis last year of the most difficult exam found that during a six-month period, only 41 percent of black and 46 percent of Hispanic candidates passed the test their first time, compared with 64 percent of their white counterparts.”
This is a paradox, as two policy goals conflict: to diversify the teaching force and to make teacher certification exams more difficult to pass. It seems likely that aligning teacher exams with the Common Core will worsen the problem.
Read Alan Singer here on the uselessness of standized tests for teacher certification. He writes that such tests are notoriously unable to predict who will be a good teacher and who will be a bad teacher.
He recommends remedies, beginning with FIRE PEARSON.
He concludes:
“There is no foolproof way to evaluate prospective teachers or anybody else for that matter. People have bad hair days and perform below expectation. Life also interferes with work and sometimes people do not develop as expected. Just look at some of the high draft choices in professional sports. There needs to be support and evaluations along the way and alternative career paths. There is no inoculation or test that can be administered at the start of someone’s career that will ensure people will be great teachers down the road.”
Diane Ravitch's blog 
I just had to hand over my social security number and other personal information to Pearson so I can take the NES test to continue teaching. WA State changed how they handled highly-qualified status, and despite the fact that I was considered “highly-qualified” prior to this, due to an incompetent HR person, I now must take a racist, invalid test to “prove” that I am highly-qualified. If I cannot pass this test, I cannot teach, even though I have taught, successfully, for 17 years.
The NES has a long history of scoring issues and lawsuits. Boston and Atlanta are two that come to mind, where the NES was used as a gatekeeper and incorrectly prevented numerous African-American educators from getting their certification.
The irony is not lost on me that I am now in the same boat as my students. My future depends on a worthless, invalid test.
LikeLike
So exactly how are standardized tests biased against blacks and hispanics?
LikeLike
Standardized tests are normed on a bell curve. The most affluent students dominate the top half. The least affluent cluster in the bottom half. Figure out how that affects black and Hispanic teachers, Thom.
LikeLike
That absolutely stinks. I’m really sorry to hear that. In Utah, those of us who graduate university before a certain date don’t have to take those tests, but the tests are garbage nonetheless. I teach geography, and my sister in law recently sent me a question from a Praxis practice exam that had an incorrect statement in the question itself. No one should have to “prove” their competency on something that is biased and wrong.
LikeLike
I was in that category before – having graduated a long time ago – but agreed to take on a non-core assignment (within special ed) this year. Thus I did not end up in the “highly-qualified tool”, otherwise known as a database, that our state ed department (OSPI) is now using. Our HR department, which has known about these changes for 4 years, didn’t inform any of us. So now all of our elementary librarians, PE teachers, district instructional staff, etc. all have to take the test. Nice way to treat teachers, no? Of course my district’s HR person won’t take any responsibility for screwing up, and claims lots of other districts are in the same boat, but when I called around to a couple of ones nearby, and they not only knew about it, but made sure teachers could move around to be covered. A couple of teachers in my district just decided to quit rather than take the test. Another one went to a private school. A lot are retiring early, then will sub to make a livable income until they are 65 and can draw full retirement.
LikeLike
So it would seem that the real question whether this would become a non-issue with a better/different selection process?
Tests like Praxis don’t seem to be a great way to cull the herd, but fairness is not the primary criteria. Selecting those who have the best chance at becoming strong teachers should be. Or culling out those who won’t cut it.
Maybe the deeper question is at what point(s) do we cull the herd: on entering programs, during the program, or after entering the classroom? The third choice is what we’ve been doing for years.
LikeLike
I think more needs to be done during the program. I have had a few student teachers: 3 from our local public university, 1 from a for-profit program (not my choice). 2 out of the 4 were really not cut out for the classroom. When I reported this to their supervisors, the reactions were very different. For the student at the public university, the supervisor increased her visits, and then concurred with me. She was pulled and asked to rethink her decision, take some more classes, and she did eventually agree that teaching was not for her. The for-profit program first asked me to reconsider, then blamed it all on me, and moved the person to, as they put it, “a more suitable placement”. She did make it through her student teaching at her next place, but the mentor teacher was feeling a lot of pressure from the for-profit group to push her through, and they even made her rewrite her letter of “recommendation”. I had heard that she got a job but was not renewed for another year. She should not have been teaching.
In my own program, there were two people asked to leave right away, and a third who was asked to leave the program and try again the following year.
LikeLike
Two of the most important reasons why standardized testing is pushed so hard by the leaders of the self-proclaimed “education reform” movement is that:
1), When it comes to OTHER PEOPLE’S CHILDREN, it doesn’t just provide an excuse—under the guise of an objective measure-to-punish regimen—to mandate massive failure but it also attempts to make the “uneducables” (Rahm Emanuel) and “non-strivers” (Michael J. Petrilli) and their parents and their associated communities willingly accept and internalize the label of “failures.” *See Duane Swacker, many threads on this blog.*
2), When it comes to the teachers of OTHER PEOPLE’S CHILDREN, the same sucker punching scam comes into play.
Many other things are possible if people accept the “results” of this rigged game. The title of this posting points to one of many.
Before action is possible, people need to come to the same realization as a genuine American hero:
“I freed a thousand slaves. I could have freed a thousand more if only they knew they were slaves.”
Or as another genuine American hero put it:
“I didn’t know I was a slave until I found out I couldn’t do the things I wanted.” [Frederick Douglass]
Once people realize they are being asked to welcome and comply with their own beat down, they need to take action:
“I prayed for twenty years but received no answer until I prayed with my legs.” [Frederick Douglass]
Get the word out; gives as many legs as possible to the opt out movement.
Will there be self-interested critics that furiously condemn the move from measure-and-punish to a “better education for all”? Will they engage in all the smear, jeer and sneer that only billionaire-backed movement can buy?
Of course. But as the owner of this blog and many commenters have stated, in the best American tradition we need to stay the course:
“I prefer to be true to myself, even at the hazard of incurring the ridicule of others, rather than to be false, and to incur my own abhorrence.” [Frederick Douglass]
But what about compromise? Aren’t I asking for too much? C’mon, when someone like Paul Cunningham is willing to go from monitoring Diane Ravitch to actually asking folks like her to be civil, isn’t it possible to find a Common Core of agreement?
“I am aware that many object to the severity of my language; but is there not cause for severity? I will be as harsh as truth, and as uncompromising as justice. On this subject, I do not wish to think, or to speak, or write, with moderation. No! no! Tell a man whose house is on fire to give a moderate alarm; tell him to moderately rescue his wife from the hands of the ravisher; tell the mother to gradually extricate her babe from the fire into which it has fallen; — but urge me not to use moderation in a cause like the present. I am in earnest — I will not equivocate — I will not excuse — I will not retreat a single inch — AND I WILL BE HEARD.”
I am going to state the obvious but it needs to be made explicit: a proven American hero like William Lloyd Garrison would be in the opt out movement. Speaking in plain English what needs to be said. And calling for a “better education for all.”
Lakeside School for everyone. What’s good enough for Bill Gates and his children is good enough for everyone else. No excuses. No exceptions. Whatever it takes.
😎
LikeLike
Considering a large percentage of teachers don’t teach math, why would they need to be good at hard math questions? How about the science part, the extensive knowledge of history, knowledge of aircraft engine systems? What should really be on a test?
LikeLike
I’m sure a lot of people who can score well on calculus tests make very poor primary math teachers.
LikeLike
Why is it so hard for folks to grasp the idea that a test score is just data? Considered out of context a test score has no meaning or value. Assigning meaning and value to a test score is an act of oppression.
LikeLike
“Assigning meaning and value to a test score is an act of oppression.”
That bears repeating.
LikeLike
Since posting this, I’ve refined my thinking a bit… Assigning meaning and value to a test score – without regard to or understanding of context – is an act of oppression.
LikeLike
The Common Core with its inappropriate content and rigged cut score is indeed oppression against students, teachers and especially children of poverty, and classified and language minority students.
LikeLike
If I may correct your statement: “Why is it so hard for folks to grasp the idea that a STANDARDIZED test score is just INVALID data.”
LikeLike
A false meme from the Queen of False Memes:
“Teachers who are not themselves well educated are not going to go on to educate their future students to the levels that we need,” said Kate Walsh, the president of the National Council on Teacher Quality.
We had a vacancy for a French teacher at my school several years ago, and hired a young woman on a waiver. She was a legal immigrant (an immigration lottery winner) from Bulgaria, had studied at a French school and spoke several languages quite well (including English!). Her training at a pretigious university in Europe had trained her well in her subject area and her knowledge of second language acquisition helped her to smooth the path for the 2/3 of our students who were working in English as a second language. She continuously improved in her classroom practice, was a contributing faculty member and a role model for our kids.
She could not pass the Mass. teacher certification exam because it contained archaic language from the Federalist Papers.
When she ran out of time to get a waiver, she was hired by a private university, where she successfully taught French and English as a Second Language for several years.
Who were the losers?
LikeLike
I want to cautiously approach a response to this, lest anyone misunderstand what I mean to say. First, I do not believe that African American or Hispanic teachers or students are inferior intellectually to anyone.
My dilemma – On one hand we say that African Americans and Hispanics are receiving an inferior education to that of many white people, so how can it be expected that African Americans and Hispanics, with inferior preparation, would score as well as better prepared whites?
Why is it that we state inferior education as a fact and then we blame the disparate scores on the test, saying that it is biased against minorities?
I truly wish an answer to this. I am not commenting on the validity of a test to determine who will make good teachers.
Scientifically, not emotionally, what is the bias in the Praxis test?
LikeLike
When I was an undergraduate a century or so ago, NY also had an exam that restricted entry into the teaching profession. Only it was then an oral exam. Administered by the then-dominant Irish school authorities, it had the effect of marginalizing Jewish candidates who spoke wid a hacent, like Lawn Gyland and toity-toid. What our accents might have had to do with our classroom effectiveness always baffled me.
Nevertheless, to deal with this problem, NYU required that liberal arts undergraduates take a year-long speech course. We’d not only learn not to deploy our accents, but we might also learn to project, to use presentational materials (I employed the classroom window-pole to illustrate lances), and to outclass the examiners. The reigning theory was that it was the responsibility of our education to help us overcome whatever “limitations” we might have brought with us into a classroom.
These high-stakes tests clearly do not help much, if at all, toward realizing that goal. Let teachers and students (and not the Regents or school managers or ignorant philanthropists or self-interested publishers) figure out what would be helpful. Keeping in mind that the goal is not to eject teachers but to help them become more successful.
LikeLike
This will seem petty in light of the seriousness of the divide but Asians are a minority group. Their success in academic settings is taken as monolithic. The struggling Asian student then has the additional burden of being a societal minority with an almost crippling academic stereotype to surmount. Filipino isn’t Chinese isn’t Korean isn’t Thai isn’t Vietnamese isn’t……Indian isn’t Pakistani isn’t Nepali isn’t Mongolian.
The intra-family pressures on Asian students, who are the walking talking retirement plans for their parents, isn’t visible but can be omnipresent. Please do not forget that Asians are a minority in this country and that academic success can be as a burdensome a stereotype as lumping all categories of people together.
LikeLike
racialdivides: you are partially restating something that the late Gerald Bracey brought up as one of his 32 principles of data interpretation in his READING EDUCATION RESEARCH: HOW TO AVOID GETTING STATISTICALLY SNOOKERED (2006):
“4. When comparing groups, make sure the groups are comparable.”
I agree with your points and simply add [based in part on firsthand experience] that in just taking those of Japanese descent in current US society, you are not getting a representative slice of what Japanese people, in the whole of Japan and of all sorts of backgrounds, are like. Like an large group of human beings anywhere they can offer examples of every type of human behavior and accomplishment, good or bad.
An example of how comparisons can be misleading from the above book, p. 7:
[start]
5. “When I compare our high schools to what I see when I am traveling, I am terrified for our workforce of tomorrow” (Bill Gates, in a speech to the nation’s governors, February 2005). Any critical remarks?
Sometimes statistics are not needed to identify a silly comment. Let us suppose Gates lands in Lyons, France, and his hosts want to show him a high school. How often do you suppose the host would say, “Ah, monsieur Gates, we have zees really bad high school we wish you to zee. It is really ’oreeble.’ Right.
I once inquired about observer access to Japanese schools and was told by an American researcher who had lived in Japan off and on for a number of years, “Look, the high schools in Ozaka are ranked one to twenty-seven. You can get into one and two, maybe even twelve or thirteen. Not even Japanese researchers can get into twenty-six and twenty-seven.” It is most unlikely that Gates’ foreign hosts wish to show him even a typical high school, much less a bad one.
[end]
And, just speaking for myself, I much appreciate you bringing these points up, particularly the all-too-often facile equivalencies made between such disparate groups.
😎
LikeLike
Fifty-one years ago this month I married into a working-class Black family from my own working-class White family. My wife’s intelligence is always on display for all to see. but give her one of these tests and she’ll score low. I score extremely high on them and a friend of ours scores off the scale. So what? My friend hasn’t achieved one/tenth what my wife has achieved but he can discuss theory till the cows come home. I know exactly where my wife messes up on these tests, these tests do reveal how well acculturated you are into the dominant culture. If that’s what we want for teachers but just with darker skins, why don’t we end the de facto segregation in our society so everyone will turn out like the people in Seinfeld? Years ago I took part in a school skit as a character in Seinfeld and had no idea who this was. I’d never seen Seinfeld. The next week the NYT listed favorite TV programs by race: Seinfeld came in first among Whites and 56th among Blacks. Why can’t people look at a fact like that – “empirical” as they like to say – and deduce that we have different cultures in this country and that determines the pattern of their responses on standardized tests?
LikeLike
pbarret: thank you so much.
If I may, riffing off of your comments as well as referring to a 2006 work by Gerald Bracey mentioned by me in an above comment, #23 of his principles of data interpretation:
“If a situation really is as alleged, ask, ‘So what?’”
I think you would enjoy, and benefit from, reading his book, for just like you he adds immediately after enunciating this point:
[start]
This principle is not an invitation to unthinking skepticism. You should then seek the “what” in “So what?” For example, if American eighth graders and tenth graders are in the middle of the pack in international comparisons, what are the ramifications and implications?
[end]
And, if I may, when talking about the wonderfully diverse talents and abilities that every single person has—and the sad and pathetically narrow slice of that measured by standardized tests of any variety—I am particularly fond of this quote from a certain Albert Einstein:
“Everybody is a genius. But if you judge a fish by its ability to climb a tree, it will live its whole life believing that it is stupid.”
BTW, whatever happened to that fella? Did he ever do anything of note?
😎
LikeLike
Thanks for the great insights for what is wrong with standardized testing. They fail to measure creativity, perseverance, resourcefulness, organization, a sense of humor and most of all character and humanity. Testing does not improve outcomes for students, and they may be harmful to some. They waste our time and energy, and our time would be better spent teaching.
LikeLike
Didn’t get an answer to my last question. How are standardized tests biased against blacks and hispanics? Ravitch claim that: “Standardized tests are normed on a bell curve. The most affluent students dominate the top half. The least affluent cluster in the bottom half.”
But isn’t that the point of the test? To discriminate between high, medium and low achievers?
LikeLike
Thom,
Let me take a stab at answering your question. I am white, Jewish, middle class. My mother held an MA and my father had some college education. My parents read three daily newspapers and two weeklies. I attended Hebrew school and took piano lessons. I had swimming lessons at the YMCA and attended free art classes at the Newark Museum. My parents did everything within their limited financial means to further my education. We took family vacations. I had plenty of books and toys. I was read to every night even after I was able to read myself.
My students are African Americans and Hispanics growing up in poverty. Most of them have none of what I took for granted growing up. Many of their families are fractured. A few reside with foster families. Some suffer from food insecurity. A number of them have family members who are in are incarcerated. None of them have the literacy input I received from my parents and extended family. Many speak non-standard dialects. I learn about gassing up and hiking from my students. As a consequence of my upbringing, I do well on standardized tests. My students score poorly due to not sharing the cultural knowledge being tested.
LikeLike
Great answer, NJ Teacher. Poverty is the reality for many African American and Hispanics. If they are poor, they will generally not perform as well on the tests. Likewise, middle class Hispanics and African American students will do much better on standardized tests. By the way, many of the poor minority students may have great potential, but they haven’t had the same opportunities to grow and learn as middle class students.
LikeLike
But that’s a problem with society, not standardized testing. If you accept that standardized tests measure something we think of as important in some context (construct and criterion validity), then the problem is not the tests themselves. The results of the tests show that something is wrong with society, not the tests themselves.
LikeLike
Not true, Thom. Tests for drivers’ licenses are not normed on a bell curve. Standardized tests are. They are designed to produce winners and losers. They have multiple negative side effects: teaching to the test (unprofessional), narrowing the curriculum, cheating, gaming the system. If standardized tests are so valuable, why do so few private schools administer them to their students?
LikeLike
So because my wife scores low on such tests and I score I, I would be a better teacher? I wish you would expand on your thinking. Do you want good teachers or good test takers? As I tried to say in my earlier comment, I know people who score really well on tests and who can’t do s***.
LikeLike
pbarret,
Are your remarks addressed to me? No, I do not think I am a better teacher because I score well on standardized tests. I am a beneficiary of white privilege. Some of the teachers in my school share the same cultural backgrounds of our students, which can be an advantage. A successful teacher reaches out to children to advance their learning trajectories.
LikeLike
It’s a great question. There is much more to teaching than test scores. Of course, they need to know their subjects. Teachers need to be clever about designing engaging lessons. The best teachers care about students and have empathy. Teachers need to have management skills to organize their class, manage paper work, and meet deadlines. They need to know how to work with others and have good interpersonal communication skills. Teaching is demanding. They need stamina, determination and a sense of humor. How many of us have had the great researcher in college that was really poor and a bore in the classroom?
LikeLike
Thom,
Standardized tests are founded on the same principle as IQ tests. They accurately measure family wealth and income. Why not skip the test and just look at tax returns?
LikeLike
pbarret: as I see it, I think the point that others are making is that standardized tests (especially of the high-stakes variety) are both a distraction and subtraction from what really counts in life.
That’s why when it comes to THEIR OWN CHILDREN, the heavyweights in the self-styled “education reform” movement have their children in places like Lakeside School (Bill Gates), Delbarton School (Chris Christie), U of Chicago Lab Schools (Rahm Emanuel), Harpeth Hall (Michelle Rhee/Johnson), and so on.
That is why, for example, Lakeside School is busy planning to add a performing arts center to its upper campus, not a test-prep center. And their athletic facilities…
Sidwell Friends where the Obamas send their children? Look at the current schedule of events and ponder this statement of purpose concerning “The Arts at Sidwell Friends School”:
[start]
The Arts program at Sidwell Friends reflects our conviction that all our students have within them the light of creativity and can learn much of value from their own and others’ creative work. The program encourages students to discover, develop, and express their own creative impulses and talents and to celebrate the fundamental place of the Arts in human life. Through courses in visual and performance arts and opportunities to share their efforts with the wider community, students discern the historical, cultural, and social context and meaning of the Arts as they learn the importance of process as well as product. They learn to take risks, to grow from failure, and to strive for excellence. They encounter the power of creative energy and personal discipline through both individual work and commitment to an ensemble. Through these experiences, they come to understand more about Art as an expression of the Quaker belief that “there is that of God in every person.”
[end]
Link: http://www.sidwell.edu/arts/index.aspx
Opt out of the hazing ritual known as standardized testing. Opt in to genuine teaching and learning.
Just my dos centavitos worth…
😎
LikeLike
You obviously have no clue why students of particular demographics fall into the lower half of bell curve. It’s widening socio-economic inequality (e.g., poverty) across the nation.
LikeLike
Thom,
” How are standardized tests biased against blacks and hispanics?”
and
“If you accept that standardized tests measure something we think of as important in some context (construct and criterion validity), then the problem is not the tests themselves. The results of the tests show that something is wrong with society, not the tests themselves.”
I believe others have answered your first question now sufficiently.
So to get to the second statement of yours I have quoted. The supposition that “standardized tests measure something” is a false supposition. Standardized test are not “measuring” devices and that “something”-the teaching and learning process, no matter at what level-cannot be adequately defined due to epistemological and ontological concerns, meaning that there is no “construct and criterion validity.
Noel Wilson has proven the COMPLETE INVALIDITY of the educational standards and standardized testing regime in his never refuted nor rebutted dissertation “Educational Standards and the Problem of Error” found at: http://epaa.asu.edu/ojs/article/view/577/700 Wilson points our thirteen sources of error in that regime any on of which render the process invalid.
Thom, to understand those various sources of invalidities please read Wilson’s work. I’ve included my summary of it, but it doesn’t do justice to reading and understanding the whole work.
Brief outline of Wilson’s “Educational Standards and the Problem of Error” and some comments of mine. (updated 6/24/13 per Wilson email)
1. A description of a quality can only be partially quantified. Quantity is almost always a very small aspect of quality. It is illogical to judge/assess a whole category only by a part of the whole. The assessment is, by definition, lacking in the sense that “assessments are always of multidimensional qualities. To quantify them as unidimensional quantities (numbers or grades) is to perpetuate a fundamental logical error” (per Wilson). The teaching and learning process falls in the logical realm of aesthetics/qualities of human interactions. In attempting to quantify educational standards and standardized testing the descriptive information about said interactions is inadequate, insufficient and inferior to the point of invalidity and unacceptability.
2. A major epistemological mistake is that we attach, with great importance, the “score” of the student, not only onto the student but also, by extension, the teacher, school and district. Any description of a testing event is only a description of an interaction, that of the student and the testing device at a given time and place. The only correct logical thing that we can attempt to do is to describe that interaction (how accurately or not is a whole other story). That description cannot, by logical thought, be “assigned/attached” to the student as it cannot be a description of the student but the interaction. And this error is probably one of the most egregious “errors” that occur with standardized testing (and even the “grading” of students by a teacher).
3. Wilson identifies four “frames of reference” each with distinct assumptions (epistemological basis) about the assessment process from which the “assessor” views the interactions of the teaching and learning process: the Judge (think college professor who “knows” the students capabilities and grades them accordingly), the General Frame-think standardized testing that claims to have a “scientific” basis, the Specific Frame-think of learning by objective like computer based learning, getting a correct answer before moving on to the next screen, and the Responsive Frame-think of an apprenticeship in a trade or a medical residency program where the learner interacts with the “teacher” with constant feedback. Each category has its own sources of error and more error in the process is caused when the assessor confuses and conflates the categories.
4. Wilson elucidates the notion of “error”: “Error is predicated on a notion of perfection; to allocate error is to imply what is without error; to know error it is necessary to determine what is true. And what is true is determined by what we define as true, theoretically by the assumptions of our epistemology, practically by the events and non-events, the discourses and silences, the world of surfaces and their interactions and interpretations; in short, the practices that permeate the field. . . Error is the uncertainty dimension of the statement; error is the band within which chaos reigns, in which anything can happen. Error comprises all of those eventful circumstances which make the assessment statement less than perfectly precise, the measure less than perfectly accurate, the rank order less than perfectly stable, the standard and its measurement less than absolute, and the communication of its truth less than impeccable.”
In other word all the logical errors involved in the process render any conclusions invalid.
5. The test makers/psychometricians, through all sorts of mathematical machinations attempt to “prove” that these tests (based on standards) are valid-errorless or supposedly at least with minimal error [they aren’t]. Wilson turns the concept of validity on its head and focuses on just how invalid the machinations and the test and results are. He is an advocate for the test taker not the test maker. In doing so he identifies thirteen sources of “error”, any one of which renders the test making/giving/disseminating of results invalid. And a basic logical premise is that once something is shown to be invalid it is just that, invalid, and no amount of “fudging” by the psychometricians/test makers can alleviate that invalidity.
6. Having shown the invalidity, and therefore the unreliability, of the whole process Wilson concludes, rightly so, that any result/information gleaned from the process is “vain and illusory”. In other words start with an invalidity, end with an invalidity (except by sheer chance every once in a while, like a blind and anosmic squirrel who finds the occasional acorn, a result may be “true”) or to put in more mundane terms crap in-crap out.
7. And so what does this all mean? I’ll let Wilson have the second to last word: “So what does a test measure in our world? It measures what the person with the power to pay for the test says it measures. And the person who sets the test will name the test what the person who pays for the test wants the test to be named.”
In other words it attempts to measure “’something’ and we can specify some of the ‘errors’ in that ‘something’ but still don’t know [precisely] what the ‘something’ is.” The whole process harms many students as the social rewards for some are not available to others who “don’t make the grade (sic)” Should American public education have the function of sorting and separating students so that some may receive greater benefits than others, especially considering that the sorting and separating devices, educational standards and standardized testing, are so flawed not only in concept but in execution?
My answer is NO!!!!!
One final note with Wilson channeling Foucault and his concept of subjectivization:
“So the mark [grade/test score] becomes part of the story about yourself and with sufficient repetitions becomes true: true because those who know, those in authority, say it is true; true because the society in which you live legitimates this authority; true because your cultural habitus makes it difficult for you to perceive, conceive and integrate those aspects of your experience that contradict the story; true because in acting out your story, which now includes the mark and its meaning, the social truth that created it is confirmed; true because if your mark is high you are consistently rewarded, so that your voice becomes a voice of authority in the power-knowledge discourses that reproduce the structure that helped to produce you; true because if your mark is low your voice becomes muted and confirms your lower position in the social hierarchy; true finally because that success or failure confirms that mark that implicitly predicted the now self evident consequences. And so the circle is complete.”
In other words students “internalize” what those “marks” (grades/test scores) mean, and since the vast majority of the students have not developed the mental skills to counteract what the “authorities” say, they accept as “natural and normal” that “story/description” of them. Although paradoxical in a sense, the “I’m an “A” student” is almost as harmful as “I’m an ‘F’ student” in hindering students becoming independent, critical and free thinkers. And having independent, critical and free thinkers is a threat to the current socio-economic structure of society.
LikeLike
I am not interested by epistemological relativism. If you don’t accept the premise that standardized tests measure something of importance, then fine. Just know that the rest of the scientific and educational community do accept it.
LikeLike
Thom, I come down on your side here. The questions for any item and hence any test what actually gets measured and how well does it get measured. For an item, the bottom line is how well do students who know the intended concept (judged independently) do on the item – and how poorly do students who don’t know the concept do on the item. Or – how well does the item discriminate across content. If the concept is important, almost any item gives dome information.
These discussions get sidetracked by the term “measure”. But my take is that we measure what we think kids know all the time.
LikeLike
“Epistemological relativsm”. Hey that’s a good one, ranks right up there with “post modern claptrap” “I’m glad you accept that the world is flat and the sun revolves around the earth.
What part of invalid do you not understand? Your assertion that the “rest of the scientific and educational community do accept it” is false on face value.
LikeLike
Standardized tests have a racist history (see “Left Back,” chapter 4). They measure one’s family income and education. They measure opportunity to learn. That is a truism. It applies to all standardized tests. Are you amazed that kids from rich families, on average, consistently get higher scores than kids from poor families? Does rich make you smart? No, but rich gives you access to medical care, a safe neighborhood, books and computer in the home, educated parents, and other good things.
LikeLike
So my daughter just took and passed her nursing boards. Of course she went through a tough program. But she had to take a “standardized test” as well.
Had this test measured family income she would have failed.
I think we do the same thing as the “other side” when we generalize and stereotype people or tests.
LikeLike
Peter, there are exceptions at both ends of the scale. There are very poor students who are superb test-takers; there are very rich students who fail the tests. But on average, every standardized test shows that rich kids dominate the top half of the bell curve, and poor kids dominate the bottom half. Look at the SAT report on the correlation between SAT scores and family income. The richest at the top; the poorest at the bottom. The same is found on state scores, NAEP scores, international scores.
LikeLike
I think it is one of the foundational myths of our society that we are a meritocracy. A quite useful corollary is that merit is measureable by tests. Just how useful can be determined by the nearly universal acceptance that some are more deserving because of their achievements as determined by scores. And this paradigm is reinforced because we can all cite exemplars who overcame many odds to do well on these metrics. All this has been exploited by the reformistas.
But at the base, it is a myth. Like Peter Greene’s unicorns.
LikeLike
Let me ask Peter Smyth the following since he is putting his family in that group of families whose students score low on tests due to poverty (questions courtesy of Ruby Payne)
How many times a year during your daughter’s childhood did you have to move b/c you couldn’t make the rent?
Can you give a precise number on the books you had in the home?
How far did you have to travel, on average, to get to a full-service supermarket as opposed to a convenience store?
How often did you put off needed doctor’s or dentist’s visit due to insufficient money or insurance?
How often were funds stretched to bail a family member out of jail?
Did your daughter have to make special arrangements to get a ride to college – maybe date the guy giving her the ride?
Which took precedence in your family, church or school?
How often did your daughter get in fist fights?
How many guns did you have in your home for self-defense?
How many times a year was your phone service cut off, cell or landline?
And I know this is personal and I am just putting it out there for consideration, how often did you go on food stamps?
If these questions strike you as odd, then you weren’t poor, you just didn’t have any money.
LikeLike
My, pbarret, how you get stirred up when you misread or misinterpret something. My daughter scored very high on her nursing boards, a standardized test. Her mother and I are/were educators, so not so rich. Now married she has been living off her husbands salary, a firefighter’s, with two kids.
So her case is counter to the argument that high test scores are always a proxy for SES. If they were, her SATs would have meant we were rich folks. But alas, correlation is about averages.
You lost me when you asked for precisely … Teachers don’t have time for that record keeping nonsense.
LikeLike
So let’s start with falsehoods-which to assess not only students
LikeLike
By the way, what is your definition of “epistemological relativism”? Please explain.
LikeLike
Peter,
“The questions for any item and hence any test what actually gets measured and how well does it get measured.”
There is nothing, absolutamente nada, that is being “measured”. I’m from Missouri-Show Me the following about “measure” in a standardized test:
What is the agreed upon, established specification that is being used as the measuring exemplar?
What is the measuring device involved in “measuring” using that specific exemplar?
How can it be known what the error of measurement in using that instrument is?
How is that measuring device used in “measuring” the construct being measured?
What is that construct? Is it the same thing for all the questions on a given test?
Until standardized test proponents can VALIDLY answer these very basic “measurement” questions so that those using the tests understand and agree, well let’s just say that the tests are COMPLETELY INVALID.
LikeLike
To clarify this, I’d like to repost the post I was responding to: Thom’s. Here it is:
“Didn’t get an answer to my last question. How are standardized tests biased against blacks and hispanics? Ravitch claim that: “Standardized tests are normed on a bell curve. The most affluent students dominate the top half. The least affluent cluster in the bottom half.”
But isn’t that the point of the test? To discriminate between high, medium and low achievers?”
NJ Teacher gave put some nuts and bolts on the difference life experiences of high and low scorers (BTW, I note I managed to slip a typo into my post: it should have read “my wife scores low and I score high”)
Others, all others, seem to be trying to get Thom to see something and I asked Thom to elaborate on his views. Thom conflates test scores with achievement….. or did I misread you, Thom? What I am saying, by comparing me and my wife, is exactly that test scores do NOT discriminate among achievers high and low but among the affluent, the not so affluent, and the poor. I am unaware of any material on standardized tests in general and IQ tests in particular that sees no problem with matching test scores with achievement. Kamenets, in her book The Test, says high school grades are as good a measure of potential in college as the SATs are. Are all these people wrong and Charles Murray right, that test scores can be used to accurately determine who can achieve and who can’t? To answer your question, Thom, how are standardized tests biased against Blacks and Hispanics, we’d have to pull out 50 years worth of research and cite it all to you. Is that really necessary?
LikeLike
Pbarrett and Thom,
You might read chapter 4 of my book “Left Back,” a brief history of standardized testing
LikeLike
Also reference the voluminous research by the Association of Black Psychologists. You can also reference the California case of Larry P. Vs. Wilson Riles. As one holding an M.A. in Psychology I can tell you that there are numerous tests that are constantly revised due to racial bias. An example, one test question no longer used made reference to the risks on an escalator. The problem with this item was that in the early 1970’s many poorer children did not know what an escalator was and had never seen one. All tests are built with certain assumptions. The quest to find Q, pure intelligence is Quixotic at best. Common experiences and inferences of language and experience are measured in many I.Q. tests. The problem is that the language experiences we consider common are not, our measure us faulty, and we should recognize the tests are written by mostly white people with a college degree. You could say bias is part of their DNA. Does this help?
LikeLike
“we’d have to pull out 50 years worth of research and cite it all to you. Is that really necessary?”
Nope, it’s not necessary. Just read Wilson’s review of the AERA’s, APA’s and NCME’s testing bible “Standards for Educational and Psychological Testing”, “A Little Less than Valid: An Essay Review”
http://www.edrev.info/essays/v10n5.pdf and you’ll see why the whole standardized testing process is COMPLETELY INVALID.
LikeLike
Peter Smyth, I’ll end my excursion here b/c this is the sort of thing that happens all the time on listservs and blogs: you wrote “Had this test measured family income she would have failed.” In the context of our discussion of poverty, you clearly implied you fell into a low-income group when in fact you were teachers, an income few poor people would think of as equal to theirs. But it’s clever, b/c people will say, “See, Peter’s family was poor and yet their daughter did well on the tests, so the argument that the tests are loaded against the poor is bogus.” So I asked a series of diagnostic questions to see just how poor you were. Of course, you didn’t answer them b/c none of them apply to you. You were not poor and neither was your lovely daughter. Just what do you think low SES means – living from paycheck to paycheck? You demean poor people by comparing your situation to theirs just as you tried to demean me with your condescending “My, pbarret, how you get stirred up when you misread or misinterpret something.”
LikeLike
Pbarrett, good luck. Sounds like you have a big burr and just want to share it. How sad
LikeLike
Damn, if pbarrett has a big burr I’d hate to see how you classify whatever it may be that is so obviously irritating me. Should I assume that since you haven’t answered any of my questions in the post of 6:17 that you are diligently trying to come up with some answers?
It seems to me that the proponents of any social mechanism of assessment have the responsibility to prove the validity, appropriateness and legitimacy of said social mechanism which in this case is educational standards and standardized testing. I’ve been searching for fifteen years now to find rebuttals to Wilson’s destruction of those educational malpractices, so I would appreciate greatly any that you can offer up. I’ve given you some questions that could go a long way if answered logically and coherently towards said rebuttals/refutations. Help me out.
LikeLike