In a shocking story in Reuters, we learn that the newly redesigned SAT will have negative effects on many students–especially those who are neediest–because of the mathematics portion of the exam.
The story is part of a series.
Renee Dudley writes for Reuters:
In the days after the redesigned SAT college entrance exam was given for the first time in March, some test-takers headed to the popular website reddit to share a frustration.
They had trouble getting through the exam’s new mathematics sections. “I didn’t have nearly enough time to finish,” wrote a commenter who goes by MathM. “Other people I asked had similar impressions.”
The math itself wasn’t the problem, said Vicki Wood, who develops courses for PowerScore, a South Carolina-based test preparation company. The issue was the wordy setups that precede many of the questions.
“The math section is text heavy,” said Wood, a tutor, who took the SAT in May. “And I ran out of time.”
The College Board, the maker of the exam, had reason to expect just such an outcome for many test-takers.
When it decided to redesign the SAT, the New York-based not-for-profit sought to build an exam with what it describes as more “real world” applications than past incarnations of the test. Students wouldn’t simply need to be good at algebra, for instance. The new SAT would require them to “solve problems in rich and varied contexts.”
But in evaluating that approach, the College Board’s own research turned up problems that troubled even the exam makers.
About half the test-takers were unable to finish the math sections on a prototype exam given in 2014, internal documents reviewed by Reuters show.
The problem was especially pronounced among students that the College Board classified as low scorers on the old SAT.
A difference in completion rates between low scorers and high scorers is to be expected, but the gap on the math sections was much larger than the disparities in the reading and writing sections.
The study Reuters reviewed didn’t address the demographics of that performance gap, but poor, black and Latino students have tended to score lower on the SAT than wealthy, white and Asian students.
In light of the results, officials concluded that the math sections should have far fewer long questions, documents show. But the College Board never made that adjustment and instead launched the new SAT with a large proportion of wordy questions, a Reuters analysis of new versions of the test shows.
The redesigned SAT is described in the College Board’s own test specifications as an “appropriate and fair assessment” to promote “equity and opportunity.” But some education and testing specialists say the text-heavy new math sections may be creating greater challenges for kids who perform well in math but poorly in reading, reinforcing race and income disparities.
Among those especially disadvantaged by the number of long word problems, they say, are recent immigrants and American citizens who aren’t native English speakers; international students; and test-takers whose dyslexia or other learning disabilities have gone undiagnosed.
“It’s outrageous. Just outrageous,” said Anita Bright, a professor in the Graduate School of Education at Portland State University in Oregon. “The students that are in the most academically vulnerable position when it comes to high-stakes testing are being particularly marginalized,” she said.
College Board CEO David Coleman, the chief architect of the redesign, declined to be interviewed, as did other College Board officials named in this article.
Read the rest of the article, which contains more detail.
Some states plan to use the SAT as a graduation exam, which should not happen because the test was not designed as an exit exam but as a measure of college readiness. In the past, testmakers would warn states against misusing their test, but this is apparently not happening now. The College Board is supposed to be a nonprofit, but the SAT is its biggest money maker. Now that nearly 900 colleges and universities are test-optional, meaning that students seeking admission to not need to supply either SAT or ACT scores, the College Board has to maintain its revenues and does not warn about the misuse of the SAT.
What will those states that use the SAT as a high school graduation test do when half the seniors can’t “pass” it? What will the young people who can’t get a high school diploma do?
Diane Ravitch's blog 
I fail to see anything shocking. All the cut scores are set in the same manner to ensure a large population of failing students. We are rapidly evolving into a two class society of haves and have nots. The tests are rigged to penalize children from low income communities? What a novel concept!
LikeLiked by 1 person
The tests are also ever more rigged to create the need for: MORE TESTS.
LikeLike
And test prep.
LikeLiked by 1 person
Abigail Shure and ciedie aech: much said in few words.
To rephrase: the tests are designed and produced (within small margins of error) to ensure failure sufficient to guarantee continued, and growing, $tudent $ucce$$.
And as for standardized testing somehow leading to more social mobility and fairness and equality—rather than to a sclerotic society that advantages the few over the many—I refer all to Yong Zhao’s WHO’S AFRAID OF THE BIG BAD DRAGON?: WHY CHINA HAS THE BEST (AND THE WORST) EDUCATION SYSTEM IN THE WORLD (2014).
Thank y’all for your comments.
😎
LikeLiked by 1 person
#Abigail Shure: actually, tests like the ACT & SAT do not come with “cut scores.” Any such scores would be imposed by some outside group or agency. This isn’t a state-administered test like the NY Regents Exam, which does impose its own cut scores and can play all sorts of political games with those (and it has in the past).
Tests like the SAT take raw scores, convert them to a common scale so that sections with disparate numbers of questions can be compared (generally verbal sections have more questions in total than math sections). And the intension is for the percentile rankings on that common scale are “normal” in the statistical sense, with about 68% falling within one standard deviation of the mean, etc.
In theory, students who have taken the SAT within the same five-year period are supposed to be comparable. It’s a reasonable idea, but like all College Board and ETS practices, needs to be scrutinized. One outcome of the 5-year notion is that scores that are more than 5 years old from a given individual are no longer considered valid. When I took the GRE in 1991 to apply to grad school in mathematics education, I could NOT use my 1973 scores, posted right after I graduated college. That was to my advantage, as my math score in 1991 was 780 out of 800, as opposed to 640 back in 1973 when I was a literature major who took zero math in college.
Let me also address here the issue of time: in high school, having basically slept through my last couple of years of math, I scored a 640 on the Math SAT the second time. And time was my downfall. I just didn’t have enough math at my fingertips. I had to take “long-cuts” to do some problems (meaning that my approach was probably more time-consuming than what an average student who was decent in math would do). There were things I knew but didn’t have right in the forefront of my thinking. So it wasn’t that I got so many problems wrong so much as that there were simply a lot I didn’t get to.
Take this example: (45)(45) + 2(45)(55) + (55)(55) in my high school days (1965-8) could not be solved with calculators. They didn’t exist. And when they did exist shortly thereafter, they weren’t allowed on these tests. Not until 1995 were they allowed. So how would you do that question in a timely manner? One clever thing to note is that the total has to end in 0 (you might be able to figure out how to see that almost immediately). If only one or two of the answer choices ended in 0, that would make this either a sure thing or nearly so. But in fact, they ALL ended in 0 (that strategy can often pay huge dividends in saving time, but not here). So, I guess we have to BY HAND do a bunch of multiplication and then an addition of three big numbers.
Well, no. Let x = 45 and y = 55. Then the above is equivalent to x^2 + 2*xy + y^2. That is a very important algebraic identity, namely (x+y)^2
And since 45+55 = 100 this is 100^2 which is 10,000. Takes far longer to explain than actually compute IF you see the underlying algebraic identity. Otherwise? Start crunching numbers.
If that happens enough, you lose precious time, you can’t do all the problems, you don’t have enough time to think a little longer on some of the harder or trickier ones.
I have not gone through all the math on the new practices SATs available free for download, but what I did go through did NOT strike me as harder or more time consuming than what I had seen prior to the release of the “New” SAT. I obviously can’t say what is true for what I DIDN’T look at yet, and those complaining about the time issue might be completely right. But I can’t really go with what typical students say. Students nearly ALWAYS think the math is “too hard” and that it’s “too time consuming.” Except for those who are really well-prepared to shine on the demands of the format and content. In 1967, I wasn’t. In 1991, I was TOTALLY on top of what mattered for such tests (note that the general math sections of the GRE are no harder than those on the SAT; only the advanced test in mathematics designed for would-be graduate students in mathematics is harder). No more using “long-cuts” and instead lots of well-developed short-cuts. Few holes, if any, in my knowledge of the relevant math. A much better approach to the thinking needed for success.
None of the above is an endorsement of the New or Old SAT or any other test. Just my viewpoint as someone who has been analyzing and coaching kids for these tests since 1978.
LikeLike
MPG,
Some districts/states require a given score on SAT or ACT for high school graduation, so in effect they are picking a “cut score.”
LikeLike
MPG,
I also took the GREs twice. The second time, I nearly flunked the writing section. The first time, there was no writing section! It was my understanding that colleges seek students within certain test score ranges. Does that not function as a cut score for that particular institution?
LikeLike
Abigail. It’s a safe bet to say that colleges use the test scores differently. Some don’t use them at all.
For instance, test optional colleges and universities, more than 870. Just counting four year institutions, according to infoplease.com, there are 2.474. More than a third are test optional.
http://www.fairtest.org/university/optional
Then take a look at what Stanford has to say about what they look for in first year undergrads that apply.
Stanford says, “There is no minimum GPA or test score” It’s the only mention of tests in their selection process overview that runs long in six sections.
https://admission.stanford.edu/basics/selection/
When our daughter applied to Stanford she freaked out about the SAT because her scores were slightly below average. I called Stanford and was told that test scores were not as important as other factors they felt were more important. She got in and graduated in June 2014.
I think the SAT and ACT is a business, and they have done a good job overselling their importance, a lot! Just like big tobacco, big oil, big sugar and big pharma do. And they almost always cherry pick facts, mislead and lie. To keep making bigger profits they have to.
LikeLike
@booklady, I said explicitly in my first paragraph that any cut scores are set by outside groups or agencies, then contrasted the SAT with the NY State Regents. Using the SAT as the equivalent is a STATE decision and the state, not the College Board, would decide on any sort of cut score.
@Abigail: When I took the GRE last, in 1991, there was no writing section. There was Verbal, Math, and Logic. I’ve not looked at the test carefully since then. Perhaps they dropped the latter and put in some sort of grammar. However, the word “flunked” seems like a strange word choice. You can’t fail these tests as far as they are used for admissions. You can just score what you score and see how a given graduate program views your application with that as one factor.
And since every school is free to set its own standards, including not considering the scores on a standardized test at all, calling these “cut scores” simply isn’t accurate. A ‘cut score’ by definition sets a minimum needed to pass or be given some sort of degree or diploma.
Yale might have an average GRE score, but there will be students well below that who get admitted and those with higher than average scores who are not admitted. That’s not a cut score. I know of no college or graduate school with admissions policies quite that rigid.
I’ve told students for decades: virtutually everyone in America can get into college. What high test scores do is expand the range of colleges you are likely to get into. But there is no score that guarantees admissions to a particular school, particularly as you look at the most selective institutions.
LikeLike
Sadly, this doesn’t surprise me.
LikeLike
Aren’t overly wordy math problems emblematic of Common Core math?
LikeLike
@HardyBoy: You mean of some of the textbooks that say that they are aligned with the Common Core Math Standards? Because: 1) there is no such beast as “Common Core Math.” There is a set of content standards which are rather terse (though there are a very large number of them, since they are supposed to cover pretty much all grades and a lot of different main strands of mathematics), stipulating specific things students are expected to know and/or be able to do at a given grade. We can argue against the very idea of such standards (which I’ve done for years), but once you have standards, this is what they usually look like when they are done at the governmental level. Look at various state standards for math prior to the Common Core. Same sort of thing.
2) nothing in those standards stipulates anything about how problems are supposed to be written – lots of words, few words, or anything else. But the major publishers in their infinite idiocy, have published some rather doubtful texts and workbooks (and tests) that have become the basis for widespread wastes of time (plenty of it mine) debating whether the Common Core Standards demand that students represent 5×3 as 3+3+3+3+3 rather than as 5+5+5 and if they do the latter, they are marked wrong. Note, that there is no actual problem shown in the video (check YouTube for “Why is 5+5+5=15 Wrong on Common Core Test”), just someone outraged over an alleged test problem. We don’t know what test or what the question was. And we could argue from now until Doomsday without knowing anything at all, because I can assure you that nowhere in the actual standards does it even vaguely address this issue other than to correctly state that the natural numbers, integers, rational numbers, and real numbers all have a commutative property for both addition and multiplication. Hence, 3×5 = 5×3. Any math teacher who marks a student off for saying so is silly, although of course in a real-world context, 3 five-dollar bills are probably preferable to 5 three-dollar bills.
LikeLike
All students are harmed by all standardized tests. Ranking students by test scores hurts the high scorers as much as the low scorers. They just tend to be less aware of the harm.
LikeLike
SAT calls itself a non-profit.
This is how non-profit they are: David Coleman, one of the leading manipulators/movers behind the destruction of the highly successful up to NCLB, community based, mostly locally controlled, democratic, transparent, [real] non-profit, traditional public schools earned nearly $750,000 in 2012 when he took over the College Board (SAT).
In 2012, the median compensation for the CEO of a non-profit was $120,396.
Click to access 2014_CEO_Compensation_Study.pdf
Not-for-Profit College Board Getting Rich as Fees Hit Students
In 2011, the former CEO “turned the nonprofit company into a thriving business, more than doubling revenue to $660 million by boosting fees …”
http://www.bloomberg.com/news/articles/2011-08-18/not-for-profit-college-board-getting-rich-as-fees-hit-students
In August 2012 revenue from AP tests was almost $300 million.
http://bulletin.represent.us/sat-collegeboard/
It’s obvious that non–profit status has been hijacked by for-profit pirates and carpetbaggers.
LikeLike
Why did colleges go along with this system in the first place?
They use standardized tests to determine placement in remedial courses, too. There’s several studies that indicate those tests are garbage and they’re putting WAY too many students in remedial courses.
Why do they hand so much power to these testing companies? One would think they would have more confidence in their own ability to make these judgment calls.
I don’t agree that we should measure people solely on test scores, but if we’re doing that we should at least tell students the truth. We rank on standardized test scores. That’s what we do. Denying it with this nonsense about how they’re “more than a score” is just dishonest. They know it’s a lie. We’re not fooling them.
LikeLike
I’m sure teachers know this but have you ever listened to high school students compare scores among themselves? They buy into this completely. They believe this is the sum total measure of what they can do for the rest of their lives.
It’s such a shame, because obviously the students who score poorly for whatever reason pick this up. It just seems nuts to give testing companies that level of what is almost societal control. Someone who scores poorly has to OVERCOME this huge assumption that they probably won’t do well in anything.
LikeLike
humans do not learn in the same way on the same day
Let’s be sure to honor only those who receive the most support and those who develop sooner. It would be a shame if someone without a lot of support and who developed slower would be allowed to grow and become more useful than those who met some bloated ego test developer’s notion of what is important.
LikeLike
More colleges should reject the SAT and create their own local admission standards. The GPA is a better indicator of college potential than the SAT. The new SAT is even more discriminatory than the old version. With the math section so dependent on a student’s command of language and reading skills, it places ELLs and LD students at an even bigger disadvantage than before.
LikeLike
Not a surprise at all that the SAT is a challenging test for disadvantaged students, but what I am shock at is that school are requiring it for graduation. SAT is designed to benefit the richer, upper class families. They have money/time to funnel resources to their children like SAT tutors and prep books (all $$$). The test was always about assessing one’s test taking abilities, not about their actual intelligence. By using the SAT as a graduation exam, you are taking the chance at a diploma away from ELLs, disabilities, and they kids who have to work in their free time. It’s all about who has the money.
LikeLike
A similar situation with the the new common core aligned state test in PA. Because the math test is so concerned with assessing critical thinking, almost every math problem is presented in word problem form. These word problems are often so difficult to understand, that whatever math skill is being tested gets lost. I know that the designers of the test think they are structuring the problem to assess critical thinking. But the word problems end up being overly complicated. Critical thinking has it’s place, but it’s not everything. Knowing how to add, subtract, and simplify fractions are real skills. Not every math has to be buried in complicated word problems nuanced (supposedly) to assess critical thinking. For students with strong math skills but weak reading comprehension, this is a major handicap. Even teachers are confused by these word bound questions. The math test, in effect, becomes a reading comprehension test.
LikeLike
I think one mistake they make is believing that critical thinking in math can only be assessed through word problems. They skip the beauty of numerical patterns that really do not need elaborate verbalization. I think it a mistake to think that one must be able to describe the universe, or just every day existence, through the lens of language.
LikeLike
I agree. I also wonder if critical thinking is something that shouldn’t be obsessively and explicitly taught. There’s nothing wrong with providing opportunities to learn and use higher order thinking , but standardizing and bottling critical thinking pedagogy and force feeding it to children just ain’t natural. Higher order thinking skills are natural and indigenous to the developing mind. These qualities of thinking are skills that can be engendered and supported in many ways, not just with the heavy handed dogmatic methods of 21 century, common core aligned, college ready crap that us teachers and our students have to suffer.
LikeLike
Agreed.
LikeLike
The problem of the quantity of text in math problems is just the tip of the first iceberg that is obscuring the second. One thing we know from past experience with the CC$$ is that in order to make the associated language tests seem more challenging while not obviously using inappropriate lexile levels or other types of developmentally inappropriate material, they chose to make the language itself more convoluted and inscrutable, they chose to write questions and to use language in ways that no one uses to communicate. They built a Tower of Babel. In the testing and instructional materials in both math and ELA that I have seen that claim to be aligned with the CC$$, this is quite evident. I have sat at a table with folks who are “high performing” in ELA and math and we have often found it impossible to ascertain what the intent of the questions are, and where in the text answers or anything that might lead to them could be found. This is more than just a short term problem for those taking tests as in a big picture view, we are teaching students inefficient, convoluted and ultimately ineffectual ways of communicating their ideas, and before that, of formulating those ideas in the first place. This is a problem that could make real the fearmongering of the biased report “A Nation At Risk”.
LikeLiked by 1 person
Tower of Babel. Perfect.
LikeLike
Excellent description of what we have seen on NY’s Common Core tests from Pearson. Fortunately NYSED has decided to offset the “convoluted and inscrutable” wording on the grade 9 CC algebra I test by setting the pass/fail cut score at +25/85 raw sore points. So in CC math, NY style, 29% = 65%. I say fortunately because algebra I is one of five Regents tests required for HS graduation.
Some of us find it rather interesting how the pass rate for grade 8, CC math (pre-algebra), is only about 30% statewide, yet by the next year, 9th graders have a near 70% pass rate on the algebra I. The psychomagicians at NYSED. find it acceptable to artificially produce widespread failure in grades 3 to 8, but magically, Common Core math and ELA scores soar in high school. Cognitive dissonance gone wild!
LikeLike
Reblogged this on David R. Taylor-Thoughts on Education.
LikeLike
Aside from the jaw-boning here about what someone’s SAT or GRE scores were (Sheesh!), the bottom line is that the SAT is an incredibly poor test. Other than predict family income – which it does pretty darned well – it just doesn’t do much. As one college enrollment expert said about using the SAT to success in college or college completion, “I might as well use shoe size.”
I’ve written on this blog numerous times that educators are doing students and public education a disservice by continuing to emphasize and rely on the SAT and its companion, the ACT, which is equally as bad. But who – exactly – has the courage to drop them? Who has the courage to say that Advanced Placement courses are for more hype and than they are educationally beneficial?
LikeLike