In this excerpt from her recent book, The Tyranny of the Meritocracy, Lani Guinier describes the tight linkage between standardized testing and family income. To the extent, then, that colleges rely on the SAT (or ACT) as a filter for college admission, they disproportionately screen out students who have not had the multiple advantages of living in affluence.
She cites data demonstrating that the SAT is of little value in predicting college performance, yet it effectively excludes students of color and students who are from low-income families.
She writes:
Close to eight hundred colleges have decreased or eliminated reliance on high-stakes tests as the way to rank and sort students. In the current environment, however, moving away from merit by the numbers takes guts. The testing and ranking diehards, intent on maintaining their gate-keeping role, hold back and even penalize administrators who take such measures. The presidents of both Reed College and Sarah Lawrence College report experiencing forms of retribution for refusing to cooperate with the “ranking roulette.”
At the center of this conflict is the wildly popular US News & World Report’s annual college-rankings issue—the bible of university prestige. In the book Crazy U, Andrew Ferguson describes meeting Bob Morse, the director of data research for US News and the lead figure behind the publication’s college rankings. Morse, a small man who works in an unassuming office, is described by Ferguson as “the most powerful man in America.” And for good reason: students and parents often rely upon the rankings—reportedly produced only by Morse and a handful of other writers and editors—as a proxy for university quality. These rankings rely heavily on SAT scores for their calculations. Without such data available from, for example, Sarah Lawrence, which stopped using SAT scores in its admissions process in 2005, Morse calculated Sarah Lawrence’s ranking by assuming an average SAT score roughly 200 points below the average score of its peer group. How does US News justify simply making up a number? Michele Tolela Myers, the president of Sarah Lawrence at the time the school stopped using the SAT, reported that the reasoning behind the lowered ranking was explained to her this way: “[Director Morse] made it clear to me that he believes that schools that do not use SAT scores in their admission process are admitting less capable students and therefore should lose points on their selectivity index.”
This is the testocracy in action, an aristocracy determined by testing that wants to maintain its position even if it has to resort to fabrication. What is it they are so desperate to protect? The answer initially seems to be that the SAT can predict how well students will do in college and thus how well-prepared they are to enter a particular school. There is a relationship between a student’s SAT score and his first-year college grades. The problem is it’s a very modest relationship. It is a positive relationship, meaning it is more than zero. But it is not what most people would assume when they hear the term correlation.
In 2004, economist Jesse Rothstein published an independent study that found only a meager 2.7 percent of grade variance in the first year of college can be effectively predicted by the SAT. The LSAT has a similarly weak correlation to actual achievement in law school. Jane Balin, Michelle Fine, and I did a study at the University of Pennsylvania Law School, where we looked at the first-year law school grades of 981 students over several years and then looked at their LSAT scores. It turned out that there was a modest relationship between their test scores and their grades. The LSAT predicted 14 percent of the variance between the first-year grades. And it did a little better the second year: 15 percent. Which means that 85 percent of the time it was wrong. I remember being at a meeting with a person who at the time worked for the Law School Admission Council, which constructs the LSAT. When I brought these numbers up to her she actually seemed surprised they were that high. “Well,” she said, “nationwide the test is nine percent better than random.” Nine percent better than random. That’s what we’re talking about….
Meaningful participation in a democratic society depends upon citizens who are willing to develop and utilize these three skills: collaborative problem solving, independent thinking, and creative leadership. But these skills bear no relationship to success in the testocracy. Aptitude tests do not predict leadership, emotional intelligence, or the capacity to work with others to contribute to society. All that a test like the SAT promises is a (very, very slight) correlation with first-year college grades.
But once you’re past the first year or two of higher education, success isn’t about being the best test taker in the room any longer. It’s about being able to work with other people who have different strengths than you and who are also prepared to back you up when you make a mistake or when you feel vulnerable. Our colleges and universities have to take pride not in compiling an individualistic group of very-high-scoring students but in nurturing a diverse group of thinkers and facilitating how they solve complex problems creatively—because complex problems seem to be all the world has in store for us these days.
I’m glad to see that the term I invented 16 years ago — “Testocracy” — is about to join the mainstream discussion of our problems. Long ago, those of us leading The Resistance debated, among other things, whether we were facing the “Standardistos” (Susan Ohanian’s coining) or a “Testocracy” (how I described it). It didn’t matter how we characterized it back than, or who was on which side (back then, Diane Ravitch was one of the most effective Testocrats, which is why many of us cheered slowly as we read “Death and Life of the Great American School System.” Later this month (January 28) when we at substancenews.net published the story of how we were sued for a million dollars and faced the demand — editorially in both Chicago daily newspapers — that I be fired after 28 years as a teacher, we will bring a bit more history and historical honesty to these discussions. Our Resistance to these absurdities began long before they became fashionable, and while it’s nice that we have more allies now, we still are aligned to the major facts and the facts of the history of our Resistance.
https://www.google.com/search?q=%22testocracy%22&tbs=bks:1,cdr:1,cd_min:1961,cd_max:1998&lr=lang_en&gws_rd=ssl
George,
I hope that you’d be happy to know that there have been little pockets of resistance going on, it’s just that they have been on a local level. And unfortunately not very successful (actually harmful for our jobs) at stemming the, what I called back in the late 90’s, “McDonaldization” of public education that put the emphasis on standards and data.
In the late 90s the district I was in decided to get out ahead of the ball with using “data driven” educational practices. I fought it then (and lost not only the battle but my position eventually). So yes, I’m sure there are many others who unfortunately experienced what I did.
Reblogged this on Kmareka.com and commented:
Great new insights from Lani Guinier on the tyranny of testing.
DO these tests mean NOTHING, though?
I grew up in… not poverty, but certainly not affluence. My father taught at a Catholic school, and my mother worked part-time at CVS. At times, we were on food stamps. Most of my clothes were hand-me-downs from older cousins.
I have always scored very well on standardized tests. Much better than my actual grades would indicate, I might add. I scored highest in my class on the SAT, and scored in the top 5%, 20%, and 25% on the three sections of the GRE when I took them.
For some of us, it’s hard to accept that these tests mean NOTHING other than family affluence, especially when we score well on the tests and grew up without much in the way of frills.
I ask mostly because from my perspective, high test scores opened doors for me that otherwise might have remained shut. I won a scholarship to college based in large part on my high test standardized test scores. Was that really just a sham?
This has come up before, werebat. I’ve never seen anyone on this blog suggest getting rid of standardized tests like the SATs and ACTs. Those are the tests that allowed you to receive scholarships and college admissions. The issues regarding year-by-year subject tests with dubious VAMs and consequences for students, teachers, and schools have nothing to do with expanding opportunities for anyone.
I also think that the fact that the SATs don’t correlate highly with college success is not an indictment of the test. I think success in college is dependent on so many variables that are not considered by the SAT. In addition, the fact that the highest correlation is with socioeconomic factors only means it is good at measuring factors that are generally only widely available to children from higher socioeconomic brackets. However, it is more of a celebration of whatever allowed you to perform so well in spite of barriers you may have faced that others diidn’t. In any case, your success went far beyond what the SAT could have predicted, but your pride in your own accomplishments probably helped you to persevere.
“I’ve never seen anyone on this blog suggest getting rid of standardized tests like the SATs and ACTs.”
Ohio Algebra II Teacher,
Perhaps you should learn to do close reading-ha ha! For I have been advocating getting rid of standardized tests for years, and on this blog basically since it began. I would hope that you have read: “Educational Standards and the Problem of Error” found at: http://epaa.asu.edu/ojs/article/view/577/700
Brief outline of Wilson’s “Educational Standards and the Problem of Error” and some comments of mine. (updated 6/24/13 per Wilson email)
1. A description of a quality can only be partially quantified. Quantity is almost always a very small aspect of quality. It is illogical to judge/assess a whole category only by a part of the whole. The assessment is, by definition, lacking in the sense that “assessments are always of multidimensional qualities. To quantify them as unidimensional quantities (numbers or grades) is to perpetuate a fundamental logical error” (per Wilson). The teaching and learning process falls in the logical realm of aesthetics/qualities of human interactions. In attempting to quantify educational standards and standardized testing the descriptive information about said interactions is inadequate, insufficient and inferior to the point of invalidity and unacceptability.
2. A major epistemological mistake is that we attach, with great importance, the “score” of the student, not only onto the student but also, by extension, the teacher, school and district. Any description of a testing event is only a description of an interaction, that of the student and the testing device at a given time and place. The only correct logical thing that we can attempt to do is to describe that interaction (how accurately or not is a whole other story). That description cannot, by logical thought, be “assigned/attached” to the student as it cannot be a description of the student but the interaction. And this error is probably one of the most egregious “errors” that occur with standardized testing (and even the “grading” of students by a teacher).
3. Wilson identifies four “frames of reference” each with distinct assumptions (epistemological basis) about the assessment process from which the “assessor” views the interactions of the teaching and learning process: the Judge (think college professor who “knows” the students capabilities and grades them accordingly), the General Frame-think standardized testing that claims to have a “scientific” basis, the Specific Frame-think of learning by objective like computer based learning, getting a correct answer before moving on to the next screen, and the Responsive Frame-think of an apprenticeship in a trade or a medical residency program where the learner interacts with the “teacher” with constant feedback. Each category has its own sources of error and more error in the process is caused when the assessor confuses and conflates the categories.
4. Wilson elucidates the notion of “error”: “Error is predicated on a notion of perfection; to allocate error is to imply what is without error; to know error it is necessary to determine what is true. And what is true is determined by what we define as true, theoretically by the assumptions of our epistemology, practically by the events and non-events, the discourses and silences, the world of surfaces and their interactions and interpretations; in short, the practices that permeate the field. . . Error is the uncertainty dimension of the statement; error is the band within which chaos reigns, in which anything can happen. Error comprises all of those eventful circumstances which make the assessment statement less than perfectly precise, the measure less than perfectly accurate, the rank order less than perfectly stable, the standard and its measurement less than absolute, and the communication of its truth less than impeccable.”
In other word all the logical errors involved in the process render any conclusions invalid.
5. The test makers/psychometricians, through all sorts of mathematical machinations attempt to “prove” that these tests (based on standards) are valid-errorless or supposedly at least with minimal error [they aren’t]. Wilson turns the concept of validity on its head and focuses on just how invalid the machinations and the test and results are. He is an advocate for the test taker not the test maker. In doing so he identifies thirteen sources of “error”, any one of which renders the test making/giving/disseminating of results invalid. And a basic logical premise is that once something is shown to be invalid it is just that, invalid, and no amount of “fudging” by the psychometricians/test makers can alleviate that invalidity.
6. Having shown the invalidity, and therefore the unreliability, of the whole process Wilson concludes, rightly so, that any result/information gleaned from the process is “vain and illusory”. In other words start with an invalidity, end with an invalidity (except by sheer chance every once in a while, like a blind and anosmic squirrel who finds the occasional acorn, a result may be “true”) or to put in more mundane terms crap in-crap out.
7. And so what does this all mean? I’ll let Wilson have the second to last word: “So what does a test measure in our world? It measures what the person with the power to pay for the test says it measures. And the person who sets the test will name the test what the person who pays for the test wants the test to be named.”
In other words it attempts to measure “’something’ and we can specify some of the ‘errors’ in that ‘something’ but still don’t know [precisely] what the ‘something’ is.” The whole process harms many students as the social rewards for some are not available to others who “don’t make the grade (sic)” Should American public education have the function of sorting and separating students so that some may receive greater benefits than others, especially considering that the sorting and separating devices, educational standards and standardized testing, are so flawed not only in concept but in execution?
My answer is NO!!!!!
One final note with Wilson channeling Foucault and his concept of subjectivization:
“So the mark [grade/test score] becomes part of the story about yourself and with sufficient repetitions becomes true: true because those who know, those in authority, say it is true; true because the society in which you live legitimates this authority; true because your cultural habitus makes it difficult for you to perceive, conceive and integrate those aspects of your experience that contradict the story; true because in acting out your story, which now includes the mark and its meaning, the social truth that created it is confirmed; true because if your mark is high you are consistently rewarded, so that your voice becomes a voice of authority in the power-knowledge discourses that reproduce the structure that helped to produce you; true because if your mark is low your voice becomes muted and confirms your lower position in the social hierarchy; true finally because that success or failure confirms that mark that implicitly predicted the now self evident consequences. And so the circle is complete.”
In other words students “internalize” what those “marks” (grades/test scores) mean, and since the vast majority of the students have not developed the mental skills to counteract what the “authorities” say, they accept as “natural and normal” that “story/description” of them. Although paradoxical in a sense, the “I’m an “A” student” is almost as harmful as “I’m an ‘F’ student” in hindering students becoming independent, critical and free thinkers. And having independent, critical and free thinkers is a threat to the current socio-economic structure of society.
By Duane E. Swacker
LOL, Duane. I confess to skimming through mosts long posts. I’ll respectfully disagree on the SATs and ACTs, but I will also read your post more thoroughly. Generally, I think we’re on the same side here.
I don’t think the tests mean nothing. I grew up in a working class neighborhood, and my family struggled. My brother and I always “had our noses in the books,” as my mom used to say. We both attended selective public schools and excelled. I think all poverty is not equal. I had an intact, loving family. My parents were responsible people with no substance problems. I grew up in the ’50s when I believe the playing field was more level for white children. It was a time of greater opportunity for all. I think it has always been more difficult for children of color.
I think “the tests mean nothing” is shorthand for “the tests don’t mean what they purport to mean often enough to justify their use.”
FLERP!: as a very succinct summary, I think you did a great job.
And, of course, I am going to do a little damage to your very terse version…
😳
I would clumsily rephrase, re my POV, as: “The tests pretend to mean a lot more than they can and prey on people’s ignorance of numbers & stats. Add in the calculated intimidation of their seeming numerical accuracy and trustworthiness. Plus a lack of discussion and understanding of what goals and definitions lie behind standardized tests, especially those of the high-stakes variety [i.e., who defines “success” and “failure” and are those valid, useful and helpful definitions]. And last but not least, when $tudent $ucce$$ for advantaged adults is factored in, you have a formula for—if “child abuse” is too strong a phrase—a toxic hazing ritual that severely damages public education and everyone in it.”
Of course, your version has the advantage of being succinct, uh, I said that already, and rolls a LOT more trippingly off the tongue [or pen or keyboard].
Thank you for your comment.
😎
Agreed! Plus they put limits on too many students, poor, LD and ELLs. They tell us nothing about character, drive, organization, resourcefulness, creativity or emotional intelligence which, according to some researchers is more important than IQ.
I am with you, but I grew up in poverty. My father was an extremely poor school teacher in India but valued education. I scored very high in GRE and was admitted to Graduate School of Engineering at the University of California at Berkeley. My post graduate education was paid for by the US Department of Energy. I am proud to say that I have made a valuable contribution to my adopted country as a scientist. There was no way for UCB to determine my ability without the standardized test results. I do not see “the tyranny of testing,” am I old fashioned, or did I miss something?
This talk about “Testocracy” is just a play on words to question changes being demanded by the general population. Parents (more educated now) are becoming more informed these days (Internet) and are demanding better performance by the public educational system. Performance can only be judged by some form of testing which has existed for a long time and will continue to exist for a long time to come.
Just like president Obama, change is what will drive the future and the public educational system must not resist change. Without change the society will stagnate.
Raj, respectfully, you are indeed missing something. No one wants to end the use of tests like the GRE. That’s not what the Testing Movement is about.
Well, Raj, these days the parents who are becoming REALLY informed are FIGHTING the testing mania. I’m opposed to the overuse of testing we are mired in right now. I question the idea that tests like the SAT, etc. have NO meaning, even though I’m aware of the correlation between student scores and family income.
It would seem that if a person grew up in poverty and still scored high on their SAT, it might say something about that person.
Testing is one form of measurement of performance. Not every career requires a standardized test for advancement or evaluation. Salesmen measure success by sales quotas; chef’s measure success by a line outside the restaurant. Sometimes success is a favorable performance review Not every task is numerically quantifiable, nor should it be. Teaching has far too many variables to be quantifiable, and if you want to measure the success of a program, as Diane Ravitch points out, you can sample.
@werebat, you’re reaching to find disagreement that doesn’t really exist. Diane and other readers of this blog don’t suggest getting rid of the SAT or standardized tests for college placement. Those tests don’t lead to punishments for students, teachers, schools, or communities. This blog is about completely different issues.
I have not seen, or do not remember, if Diane has taken a position on standardized tests like the SAT, GRE, ACT, etc. However, she regularly shares information from FairTest and FairTest argues against the use of these standardized tests. Several commenters have questioned the role of ACT and SAT in college admissions and Senor Swacker questions the concept of extrinsic assessment altogether.
Okay. I’m relatively new (maybe a year) to the site, though I read pretty regularly. I don’t recall seeing complaints about the ACT, SAT, etc. Personally I think those tests are essential, because there are huge differences in the way various schools grade students. Regardless, I can’t believe getting rid of these college placement standardized tests is a major plank of anyone’s agenda on here, and none of my many friends and colleagues who oppose the Testing Movement include these well-established tests.
A common tactic of Reformist mouthpieces is to stretch common sense beliefs to unsupportable (or at least much less supportable) extremes.
Your family valued education highly and you grew up living that value. I would have been surprised if it didn’t serve you well. Placing a high value on education and what that translates into within the family is an attribute of higher socioeconomic classes. Who are the kids coming out of poverty who are successful? I would guess we would find that they place a high value on education. For a good number that probably has translated into high SAT scores. For others perhaps their scores are just above average for a disaggregated population (since educational opportunities tend to be poorer in poor communities). Bottom line is you have every right to be proud of what you accomplished, but what you have accomplished goes far beyond those test scores. For you, those test scores are a symbol of your hard work and merit. Where we have to be very careful is in labeling students as unworthy because they did not perform well. Too many other factors may have played a factor in their performance. I think of a student I knew who was driven to succeed. He came from a single parent home with English as a second language. He went to high school in a lower socioeconomic community lacking the resources of the nearby suburban communities. He worked his tail off but could not get a high enough ACT score to gain entrance to a top tier school. His scores did help to gain entrance to a state university where I am sure he is excelling. Knowing him, he will graduate at the top of his class this June. I hope he still dreams of being a doctor and is on a path to making that a reality. He has worked harder than 99% of his peers to achieve what he has.
2old2teach: I agree that the value a family places on education has an impact. I saw that with my ELLS. My Haitians attained more education than my Latinos, even though both groups had potential. The Haitian parents understood that education was the avenue of access.
werebat73,
Anecdotal evidence notwithstanding of benefits for some from the tests, it does not a good case make to keep something as epistemologically and ontologically bankrupt as those standardized tests.
And actually your story is example A of why these tests were first invented. May I suggest that you read (if you haven’t) Nicholas Lehmann’s “The Big Test: The Secret History of the American Meritocracy” (a history of standardized testing).
Duane
“Silly Attitude Test”
I know her 2.7%
We plan to tie the knot
Relationship is heaven sent
Her SAT is hot!
“Testing Reality”
Testing’s always been a means
To keep ’em in ‘their place”
Whether for polls or simply green$
It biases the race
To werebat73, and Raj Acharya:
We need your focus on children in the age of 4 to 12 who are subjected to 9 hours of “inappropriate level of testing materials” with a set-up failure label in their young mind.
Again, at your correspondent learning skills of late teen age and well equipped with parental love, guidance, caring, motivation… to help you achieving your goal. However, could you imagine the plight of children who lack all of what you had? But, they also face the hash of testing scheme with design to fail them for corporate profit.
I do not support any scheme that intends to loot public education fund, to destroy the joy of teaching from teacher, to ruin the joy of learning from all children regardless of their circumstances of gifted, disable, normal, or above average.
In conclusion, RIGOROUS testing is approval for any designated careers to certify the professional whom people can entrust their safety in that expertise. Please join us to restore to parents, teachers and local communities their RIGHTFUL CONTROL OVER PUBLIC EDUCATION
Here is the best post to think about if you just join this website recently:
[start quote]
John Taylor Gatto received the New York State Teacher of the Year award in 1990 and was named New York City Teacher of the Year in 1991.
Here is his acceptance speech:
“The only reason I received this award ” the only reason I’ve been a GREAT teacher for my students ” is because:
I didn’t do a single thing you told me to.
I ignored your ‘standards,
I thwarted your bureaucracy and I taught unauthorized material.
I filled out those forms that said the students were in their desks, when they were really taking horizon-expanding study trips.
I had them read real books instead of those inane, dumped-down textbooks of yours,
I taught them real history instead of the porridge of revisionist pabulum you call ‘social studies’.
“Your bureaucracy is a mill that grinds up human beings and turns them into consumer fertilizer for a planned economy.
Human potential erodes as hungry minds sit in listless boredom, and teachers operate without the tools they need, just so you guys can fill your administration buildings with cushy jobs and give contracts to your cherished vendors.
“That’s why most of our students can’t read after 12 years of education ” yes, even though it only takes 3 months to learn how to read. That’s why most kids follow the herd into a bleak future instead of thinking for themselves.
“I am OFFICIALLY turning in my RESIGNATION as of today.”- JTG, 1991.
[End quote]
I hope that you accept our invitation to spread out our plight to bring back DEMOCRATIC GOVERNANCE of PUBLIC SCHOOLS in this nation. Back2basic
Thanks for that quote, m4potw!!
I hope to meet you in Chicago!
Duane
In some countries (China,for example) many of the students at the top universities are provided test prep for GRE so that they might be admitted to top US universities for grad school.
I know for a fact that test prep can make a huge difference on GRE because I took the physics subject test twice. The first time I didn’t prepare and found that I could not answer a lot of the questions because I hadn’t memorized a lot of required formulas. So I canceled my score and proceeded to memorize a bunch of formulas in preparation for the next test, which I did very well on (not because I was any genius, but because i memorized a lot of formulas that anyone with a brain could look up if and when they needed them)
Quite frankly, I don’t think the GRE indicates much. Maybe it has changed since I took it back in the 80’s, but the subject test that I took was actually fairly ridiculous.The idea that you can gauge someone’s potential to do graduate work in physics with a multiple choice test is simply a joke.
All it really shows is if you can recall a bunch of basic formulas and quickly plug in to get answers to very basic (one step) problems. That’s not real problem solving.
Maybe I’m reading too much into the words of some people, but the tone of some of the responses to my question about whether or not tests like the SAT, ACT, and GRE are completely valueless is… interesting.
For the record, I’ve been reading this blog for years, and had noticed the trends Diane talks about for years before I discovered her blog or knew who she was. I have attended one of her talks where I met her in person and had her sign my copy of “Reign of Error”, a book I purchased several copies of to give as gifts to my department as well as a small handful of local school officials and politicians (including a recent candidate for governor). I frequently write and speak locally in opposition to the corporate education reform movement, and have written several letters to local papers about the folly of local corporate reform measures — one of which got me put on a brief administrative leave (although local officials were careful to conceal that fact).
I was simply asking if some standardized tests might not have ANY value. Some of us who did very well on the things took a measure of pride in that, which might have counted for something when we couldn’t be the tallest, most popular, best looking, or most athletic in school.
I owe you an apology, werebat. It looks like you and I are largely in agreement. Where I may depart is that I think it’s a very minor issue in comparison to the major testing problems we are facing. In other words, what unites us is far more important than what divides us.
Well… I’m sorry I got miffed about it in the first place. Just didn’t want to be thought of as “the enemy” when I don’t think that’s true at all. Apology accepted.
To werebat73
It is important that we are aware of how much damage (= being failure) to be done on young mind from today’s testing scheme.
According to you, “Some of us who did very well on the things took a measure of pride in that, which might have counted for something when we couldn’t be the tallest, most popular, best looking, or most athletic in school.” Your expression confirms that the common core state standards intends to destroy PRIDE of as many as 80% of young K-2 students.
I am sorry to acknowledge that you took the personal loss to inform others about the bad intention of CCSS. However, I hope that you are happy to do the right thing for the right time whenever you walk back to your own memory lane.
My only mantra is “Good deed returns good deed, and evil follows evil”. This mantra helps me to survive in this material world by keeping doing good deeds with joy. I hope that you agree with me. Back2basic