Kenneth Bernstein regularly blogs at The Daily Kos. He is an experienced teacher with National Board Certification.
He shares his views here on test prep and testing:
For a number of years, until I decided I had ethical problems with doing so, I earned money beyond my teaching salary by teaching and later tutoring SAT prep for Princeton Review. That company probably has more experience with preparing students for standardized tests than any company in the nation. Thus it is probably worthwhile to note what they taught us to say about the SAT to our students:
Other than a bit of vocabulary and some ability to read questions accurately, what the SAT measures is how well you did on that version of the SAT on that day.
In other words, it did not measure how bright you were, nor how prepared for college you were (high school grades are a better a predictor of 1st year grades at .30), and SAT had little predictive relevance beyond 1st year grade and for that it had a predictive validity (R-squared) of around 22%, meaning that it predicted less than 1/4 the actual first year grades.
We hav always known there was a strong correlation between family income and SAT scores. Princeton review courses, which were more accessible to those with more income, merely exacerbated that problem.
Now test prep companies are already moving into the business of offering courses for high stakes state tests, and once we have tests aligned to the Common Core State Standards the problem will become even worse – those with money, who will already perform better, will gain an additional advantage by being able to purchase such test preparation.
What will this tell us? Exactly what the SATs tell us according to Princeton Review – beyond the correlation with family income, how well the testee did on that application of that test on that day.

Apologies for an error I did not pick up when I sent this to Diane. SATs are NOT as good a predictor of first year grades as are high school grades. The latter account for about .30 of the variance compared to the .22 accounted for by SATs.
LikeLike
I made the edit.
LikeLike
As a mom who shelled out the $$ for SAT prep for two of three daughters, I did it for one reason only– to get them into the schools they wanted. Never for a moment did I think they would be more prepared for college. College readiness is not a test score.
LikeLike
I probably shouldn’t do it, but when my students begin wringing their hands about SATs, I shrug and tell them that it not a measure of anything more than test-taking skills. I mention that I never took them, yet somehow I was a very successful undergrad, grad student, and professional in the career of my choice. I’m not all that old either, so my experiences are probably still at least somewhat relevant. I have been helping a student review for an SAT II subject test lately, and she discovered very quickly that it is all about decoding oddly worded questions and not about subject knowledge.
A second part of the problem is my students’ obsession with college branding; they are overly concerned with the prestige of a school rather than a good fit, and see a good SAT score as necessary to avoid being doomed to a “lesser” school. From K through grad school, education is becoming what Kurt Vonnegut once called a “micromanaged, Darwinian victory over others.” He was referring to colleges, but it has spread downward. This is the very opposite of what public education is for.
.
LikeLike
Alan,
Do you know where I can find the K. Vonnegut thoughts?
Thanks,
Duane
LikeLike
I no longer remember. I think it was in one of his essay collections rather than in one of his fictional works. Sorry I can’t be more specific.
LikeLike
The worst thing that ever happened was the idiotic U.S. News “ranking” system of colleges and universities. Absolute drivel. I wish all the institutions boycotted it.
LikeLike
Actually, all the SAT (and ACT, for that matter) measures is preparation for college. That isn’t the same thing as being able to succeed. The scores merely tell a recruiter if they have taken enough credits to be prepared for math, etc. That’s why the individual questions have different scores assigned to them, depending on difficulty. ETS/College Board has even said that is what the scores measure. It doesn’t measure future success, it doesn’t measure success in college, it certainly doesn’t measure intelligence. It simply tells recruiters the student has taken this or that course or hasn’t taken this or that course judging from the answers.
LikeLike
Yes, high school grades have always been better indicators of potential success in the first year of coll than the SATs. This was indicated in 1977 in the Wirtz Report by fmr secy of labor Willard Wirtz asked to convene panel in 75 after SATs took biggest plunge in 1 yr, 10 on verbal, 8 on math, major media declared “a literacy crisis” and Coll Bd hired wirtz to study, who took two yrs, issued study in 77 which announced that high school grades predictive validity for 1st yr coll success was 52% while SAT’s was 50%, yet Wirtz earned his fee by insisting that SATs be kept anyhow.
LikeLike
Speculation is not a foundation for new policy!
As an undergraduate, majoring in experimental & personell psychology, I had extensive training in experimental methodology & statistical analysis. One of many lessons drummed into my head back then was that even well written psychometric tools had limited predictive capabilities & a host of uncontrolled variables that could screw up your conclusions. In my letters to legislators & other political friends, therefore, I stress the following and have done so for two decades.
A well written test is capable of telling you what that student knew & did not know on the moment that test was administered. Its predictable powers are extremely limited, as there are dozens of uncontrolled variable that will influence future learning. while a well written test can tell you what the student did not know at that time, it can tell you absolutely nothing about why the student did not know that information. Making any kind of causal connections under those circumstances is the rankest of speculation. And speculation is not a foundation upon which to base new policy.
LikeLike
This hits home. I remember years ago when I took the SAT or was it the ACT? Makes no difference. When my scores came back I was disappointed and worried I wouldn’t be accepted into college. I so much wanted to go. I went to the principal. He knew me well, as he did the other students. He asked me to come into his office. He closed the door. Looked at my score. Requested my school records. He looked at them and said, ‘Kathy, who did you go out with the night before and what did you do?” I answered, No one and nothing.” I think it was then, I stopped crying. I will never forget those words. He had no doubt that I would be accepted to college and that I would make it. The rest is history! I did just that! Thank you, Mr. Hoffman for believing in me.
LikeLike
Perfect! Teachers can use their merit pay bonuses to buy prep class vouchers for the next cohort’s standardized tests. And so it goes.
LikeLike