A major new study by the National Association of College Admission Counseling (NACAC) looked at the college performance of eight cohorts of students from 33 colleges and universities. These 33 institutions do not require students to submit standardized test scores for admission. Like many previous studies, this one found that high school grades are better predictors of college success than scores on college entry examinations like the SAT or ACT.
The bottom line: Colleges and universities can learn more about future students by reviewing their four-year record of persistence and achievement rather than the results of an arbitrary test offered on one day, particularly one where some students have the means to pay for tutors to boost their scores. Colleges and universities do not need admission test scores to know which students are likely to succeed. The entry exams tend to have a disparate and negative impact on the neediest students. They are an unworthy gatekeeper. They waste the time, effort, and money of students. They benefit the testing corporations and the test prep industry, not students who hope to gain entrance to of institutions of higher education.
Here is the abstract of that study:
“This study examines the outcomes of optional standardized testing policies in the Admissions offices at 33 public and private colleges and universities, based on cumulative GPA and graduation rates. The study also examines which students are more likely to make use of an optional testing policy, and how optional testing policies can offer important enrollment and financial planning benefits. Four cohorts of institutions are examined: twenty private colleges and universities, six public universities, five minority- serving institutions and two arts institutions, with a total of just under 123,000 student and alumni records. Few significant differences between submitters and non-submitters of testing were observed in Cumulative GPAs and graduation rates, despite significant differences in SAT/ACT scores. Optional testing policies also help build broader access to higher education: non-submitters are more likely to be first-generation-to-college students, minorities, Pell Grant recipients, women and students with Learning Differences.”
Here is the summary:
“Previous research on standardized testing in admissions has examined the predictive value of testing and its fairness across widely differing pools of students. For over thirty years but increasingly in the last decade, hundreds of institutions have made admissions testing optional. This three-year study is the first major published research to evaluate optional testing policies in depth and across institutional types.
“With various forms of optional testing policies, the thirty-three colleges and universities in this study make admissions decisions without standardized testing as a credential for all students. Deliberately, the study reaches beyond the various “top 25” or “most competitive” lists. We include institutions in four categories: twenty private colleges and universities, six public universities, five minority-serving institutions, and two arts institutions, a total of approximately 123,000 student records at institutions with enrollments from 50,000 students to 350, located in twenty-two US states and territories. They vary widely, from a large scientific and technical university to a Native American college, from traditional liberal arts colleges and universities to fine arts/design institutions to urban and rural minority-serving institutions. We tried to ask, “Who is doing the heavy lifting, serving broad constituencies? Who is exploring the breadth of human intellect and promise in imaginative ways? Who is reaching out to serve students most desperately in need of access to higher education?”
“A fundamental question is: “Are college admissions decisions reliable for students who are admitted without SAT or ACT scores?” Many national educational research and philanthropic organizations such as the Lumina Foundation have presented findings to demonstrate that America will need to find successful paths to higher education for hundreds of thousands of additional first-generation-to-college, minority, immigrant and rural students, in order to grow America’s economy and social stability. This study provides the research support for optional testing as at least one route by which that can happen.
“Test scores contribute to college guidebook rankings. Perhaps equally important is self- selection by students who do not apply to colleges based on perceptions of testing or on advice from their high schools or parents. The National Association of College Admission Counseling (NACAC) “Report on the Commission on the Standardized Tests in Undergraduate Admissions” urged colleges and universities to “take back the conversation” about testing from the various groups for whom testing was either a profession or a cause. i This study is a contribution to that discussion.
“Does standardized testing produce valuable predictive results, or does it artificially truncate the pools of applicants who would succeed if they could be encouraged to apply? At least based on this study, it is far more the latter. In a wide variety of settings, non- submitters are out-performing their standardized testing. Others may raise the more complex issues of test bias, but we are asking a much simpler and more direct question: if students have an option to have their admissions decisions made without test scores, how well do these students succeed, as measured by cumulative GPAs and graduation rates?”
Here is a summary of the findings:
“With approximately 30% of the students admitted as non-submitters over a maximum of eight cohort years, there are no significant differences in either Cumulative GPA or graduation rates between submitters and non-submitters. Across the study, non-submitters (not including the public university students with above-average testing, to focus on the students with below-average testing who are beneficiaries of an optional testing policy) earned Cumulative GPAs that were only .05 lower than submitters, 2.83 versus 2.88. The difference in their graduation rates was .6%. With almost 123,00 students at 33 widely differing institutions, the differences between submitters and non-submitters are five one-hundredths of a GPA point, and six-tenths of one percent in graduation rates. By any standard, these are trivial differences.
“• College and university Cumulative GPAs closely track high school GPAs, despite wide variations in testing. Students with strong HSGPAs generally perform well in college, despite modest or low testing. In contrast, students with weak HSGPAs earn lower college Cum GPAs and graduate at lower rates, even with markedly stronger testing. A clear message: hard work and good grades in high school matter, and they matter a lot.
“• Non-submitters are more likely to be first-generation-to-college enrollees, all categories of minority students, women, Pell Grant recipients, and students with Learning Differences (LD). But across institutional types, white students also use optional testing policies at rates within low single digits of the averages, so the policies have broad appeal across ethnic groups.
“• Non-submitters support successful enrollment planning in a broad range of ways. They apply Early Decision at higher rates, increase enrollments by minority students, expand geographic appeal by enrolling at colleges far from their homes, and allow for success by Learning Difference students.
“• In a surprise finding, non-submitters display a distinct two-tail or bimodal curve of family financial capacity. First-generation, minority and Pell-recipient students will need financial aid support, but large pools of students not qualifying for or not requesting financial aid help balance institutional budgets.
“• Non-submitters may commonly be missed in consideration for no-need merit financial awards, despite better Cum GPAs and markedly higher graduation rates than the submitters who receive merit awards. Institutions may want to examine their criteria for merit awards, especially the use of standardized testing to qualify students for no-need merit funding.”

I was speaking with an upper level administrator about this issue over the weekend and was told that for low income students at my institution high school GPA looses its predictive power but ACT scores are correlated with persistence, graduation time, and university GPA. The administrator speculated that working many hours while in high school tended to lower high school GPA but not performance on a test during one Saturday morning.
LikeLike
Here is what I read at the Wash.Post on the subject. It somewhat agrees with you but not completely.
The content of the ACT is closer to the material a student has covered in high school (in fact, it is based on a national curriculum survey) than is the content of the SAT — but it is still no more accurate than the SAT in predicting college grades, research has shown. Why? Because no standardized test in which students sit there and fill in bubbles and write an essay can capture all of the work habits, coping skills, motivation and other traits needed to be successful in college.
http://www.washingtonpost.com/blogs/answer-sheet/post/what-do-sat-act-scores-really-mean/2012/09/24/33e341c0-0675-11e2-afff-d6c7f20a83bf_blog.html
LikeLike
The data about low income students comes from my institution and others may not have the same experience. I think poster Jennifer Martin brings up some very important points in the Leon Botstein thread that might interest you. Her points are consistent with my experience, which is why I like the idea of giving students alternative ways to demonstrate academic ability.
LikeLike
Reblogged this on Transparent Christina.
LikeLike
TE and others – do you know if a study has been done nationally examining the observation that the administrator made? The study mentioned here suggests that for low income students, grades are a better predictor than test scores.
There have been various challenges to the ACT’s and SAT’s. My sense, based on various studies is that overall, grades are a better predictor of college success.
Furthermore, some years ago, the ACT published a study showing that participation in debate, drama, music or student government in high school was a better predictor of adult success (as ACT defined it) than high grades in high school or college, or high scores on the ACT.
LikeLike
The observation was based on data local to my institution and was for low income (defined as Pell eligible students). This administrator had the sense that the math portion of the standardized tests was probably driving the relationship. This makes sense as the remedial and low level math courses at my institution have high DFW rates and high school GPA can contain relatively little information about the mathematical ability of our students.
I do think there is a selection problem here, but I have not had time to read the whole paper. Students with standardized test scores are likely to be making a deliberate decision about which college or university to attend. Students that make such a considered choice generally do better in college. Out of state students at my institution have a much higher retention and graduation rate than in state students for example. I also wonder about major/school choice and college GPA. At my institution the average undergraduate grade given by a school ranges from A- for the school of education to B- for the school of engineering. To the degree that students pick majors based on grades, we might well see more capable students in some majors than in others.
LikeLike
A lot is being made about test prep and tutors being used by upper-income families. I don’t see that affecting the data all that much. How does one explain that almost perfect line from the lowest income to the highest. I don’t see tutoring being applied incrementally more by the $20k-$30k-$40k households in order to establish the curve at the far left end. I would expect test prep to appear as an uptick on the right.
I think it far more likely that the results we are seeing are the result of fundamental flaws in the SAT akin to the problems highlighted by critics of IQ tests (against which the SAT correlates strongly enough for Mensa to accept).
LikeLike
xxxxxxxxxxxxxxxxxxxxxx This study explains the incredibly long, boring, highly sympathetic NYT Sun mag article yesterday (The Story Behind the SAT overhaul). Coleman is running scared.
LikeLike
TE, I think your administrator has the 2 variables reversed. Several previous studies have shown that the ACT & SAT have low predictive value for college success. There is the study done by the ACT that Joe N pointed out & the SAT also has studied one (or more). I read them years ago so I can’t cite the sources at this moment.
LikeLike
The administrator did not have the scores reversed. For Pell eligible students at my institution, a large state research university, high school GPA has no predictive power, but ACT score does.
Perhaps there is an issue of grading consistency across institutions. I noted that that one public university in the study (PU 3, my guess is UT at Austin given median SAT scores of 1300 and top 10% of class admission policy) has significantly more capable students than other universities in the study (PU5 has median SAT score of 980 and entry GDP of 2.0 in state). Would you expect the grading scales to differ in the two institutions? I would not expect them to be the same, and suspect that a B student at PU3 would be an A student at PU5. In this case, adding oranges and clementines and tangerines to the data might be obscuring the features of each. Perhaps the data should be institution and even major specific.
LikeLike
I’m so glad the tutoring edge, generally, is getting attention. This is Amanda Ripley in the WSJ, on how South Korea gets those high test scores:
“To call this mere tutoring is to understate its scale and sophistication. Megastudy, the online hagwon that Mr. Kim works for, is listed on the South Korean stock exchange. (A Megastudy official confirmed Mr. Kim’s annual earnings.) Nearly three of every four South Korean kids participate in the private market. In 2012, their parents spent more than $17 billion on these services. That is more than the $15 billion spent by Americans on videogames that year, according to the NPD Group, a research firm. The South Korean education market is so profitable that it attracts investments from firms like Goldman Sachs, the Carlyle Group and A.I.G.”
It’s the WSJ, so this private tutoring market is presented as an unqualified good, but middle class parents in South Korea are buying a “shadow education” system in addition to the public system. They don’t really have any choice. Ripley compares this to what US parents spend on “videogames” but that isn’t the choice a lot of working class people would be making, “videogames” or private market tutors. They would be re-allocating funds from things like appropriate housing or other necessities which are probably more important to the well-being of children.
Is this what we want in this country? Already-strapped working class people paying out of pocket for test prep? Why would the US Secretary of Ed ever hold this system up as a model? Celebrity tutors are not even a particularly efficient or productive use of funds for families lower on the economic scale, especially if it’s family money that is reallocated from general “quality of life” for children.
http://online.wsj.com/news/articles/SB10001424127887324635904578639780253571520
LikeLike
Thanks for publishing this on the blog. One thing which I have concluded is the education is now a full-blown public industry, with all the trapping many on this blog rail against.
Time and again contributors remind us of the evil Koch Bros, or the profiteering private schools. Yet, they seem to fawn over the status quo ‘educators’ or the vested interests such as Bill Gates, NC gov. Jim Hunt, and other Liberal luminaries.
What the rank and file teachers do not see is the snake oil used to coerce teachers that “solutions” THEY propose are best for public education.
Consider the failing programs such as NCLB and RTTT, both which enrich its creators rather than address what ever shortcomings there are. These are but national programs, but in the states are hundreds more, created by outside “do gooders”,
not the professions who know what is need, the classroom teacher.
And, the latest is Universal Pre-K, which will replace the vital nurturing inside the home with structured molding in the formal public institution. This is another untested “solution”, which will add billions more with no evidence of success, only another avenue for parents to be “drive by” rather than involved with their child’s development.
Lastly, the one thing every teacher should be fighting for is support for is discipline in the classroom which appears to be
a “third rail” due the the implication discipline of minorities has racial overtones.
Again, a long-worded response on an outstanding beautiful NC morning….ajbruno14 gmail
LikeLike
I wish this data also broke out results by gender.
LikeLike
This is nothing new. Universities and colleges have known this fact for years, but choose to ignore it to maintain a business model more concerned about rankings in U.S. News, the a fair and economical admissions system.
LikeLike
Research shows that test prep courses only gain a few points for students. It isn’t that the rich are buying the grades, The SAT is a standardized test and like all standardized tests it will follow a bell curve that matches family income and education level. The idea that the pundits are forming is that the rich have the advantage because they are buying it. but it is the same on every test. We are missing a golden opportunity to draw attention to the concept that ALL standardized testing will have the same outcome and that evaluating someones potential based on a test score is just wrong.
http://collegeapps.about.com/od/sat/f/SAT-test-prep.htm
My daughter took a course and changed her grade about 10-15 points. She told me she was one of the only kids working in the class and that the other kids were there because their parents made them go and they didn’t even try, they sat in the back and used their phones to play games. We may be able to measure whether or not someone is college ready, but whether or not they will succeed in college is determined by grit and support, not a standardized test.
But parents are being ripped off by prep companies that tell them they can raise scores. But the idea that if we just gave the low income kids a test prep course they would score higher is ridiculous.
LikeLike
Veteran Educator, you are right that standardized tests by their nature favor the most advantaged students and families.
But tutoring raises scores, and well-to-do families pay for tutors and gain more than 10-15 points. They may gain as much as 100 points.
Yes, the rich buy higher scores on the SAT and they will continue to do so.
LikeLike
As I stated above, given that the richer families are distorting the scores at the top by “buying” points, how do we explain the slope of the chart at the other end. I have not heard that those below the poverty line are investing their non-existent resources in test prep. And yet the $60k families score higher than the $40k families who score higher than the $20k families.
LikeLike
Certainly personal tutoring or a full length summer class can have a much bigger chance or raising a score and that is hugely unfair. But the average kid taking an evening class doesn’t change their score much. I did find a small study that showed close to a 100 point gain. Many sites claim more than they can produce but gladly take people’s money. I can’t wait for the whole corrupt system to collapse.
A College Board study conducted in the mid 1990s showed that SAT coaching resulted in an average verbal increase of 8 points and an average math score increase of 18 points.
A 2009 study by NACAC, the National Association of College Admission Counseling, showed that SAT prep courses raised critical reading scores by about 10 points and math scores by about 20 points
You are right as always. it is unfair either way as it doesn’t represent your potential, only your test preparation. I am just upset that I didn’t hear on any of the News Stations that every standardized test results in a parental income and education related bell curve.
LikeLike
The NACAC is the very same group that authored the study that Dr. Ravitch posted about in this entry. I would think that adds credibility to the finding of an overall 30 point gain in SAT score from prep courses.
LikeLike
The SAT and ACT aren’t predictors, anyway. They were never designed to be as such. It seems even academics have fallen for that tripe.
LikeLike
Figure 22 is interesting. In all but PU3, non-submitters of standardized test scores had higher average high school GPA AND higher average standardized test scores.
I do not know why they talk about how low average high school GPA students are more likely to submit standardized test scores. I would think a low high school GPA CAUSES students to submit standardized test scores.
Another interesting point is how the difference in GPA between non-submitters and submitters narrows between high school and college.
LikeLike
Many years ago, the SAT was called the Scholastic Aptitude Test. It was supposed to be a test of aptitude for schoolwork. That proved not to be the case. The test turned out to be a lousy measure of aptitude, so it was renamed the Scholastic Achievement Test. But it wasn’t a valid measure of achievement either. So, the College Board started calling it simply the SAT, or the Scholastic Reasoning test because it is highly correlated with g, the “general intelligence factor” measured by IQ tests. However, there are MANY problems with all such tests. See, for example, this:
http://www.slate.com/articles/health_and_science/science/2012/09/how_children_succeed_book_excerpt_what_the_most_boring_test_in_the_world_tells_us_about_motivation_and_iq_.html
The simple fact is that high-school grades are much better predictors of college success than the SAT is. And, the gains that can be made via prep for the SAT–around 20 points–are significant in today’s highly competitive college admissions environment, and that prep benefits the kids whose families can afford it.
And, of course, what the SAT does measure well is family income. It’s a GREAT predictor of that.
The SAT needs to be scrapped. Or, if it is to be retained, then it should be one of several OPTIONS that kids can chose for proving to schools their worthiness for admission. Those options could include
a. school grades
b. original work (some sort of portfolio)
c. subject-area tests
LikeLike
What the research discussed in that Slate article tells you is that the SES disparities on IQ tests and these general reasoning tests have A GREAT DEAL TO DO with disparities in motivation to succeed on tasks of this kind.
LikeLike
+1
LikeLike
I don’t know of a school that admits soley on the basis of SAT scores, so I do think it is one of the factors considered for admission.
Do you like the SAT 2 subject exams? Many of the most competitive schools require these as well.
LikeLike