This just in from FairTest:
FairTest
National Center for Fair & Open Testing
for further information:
Bob Schaeffer (239) 395-6773
mobile (239) 699-0468
GLOBAL SAT SUBJECT TEST SECURITY BREACH LAST WEEKEND;
SCORES COMPROMISED BY ADVANCE CIRCULATION OF EXAM;
QUESTIONS AND ANSWERS ALSO POSTED ON REDDIT WEB SITE
MANY HOURS BEFORE ADMINISTRATIONS BEGAN IN U.S.
Last weekend’s SAT Biology Subject Test was in circulation more than half a day before the test’s administration, according to a watchdog group that received an advance copy of the exam. In addition, test questions and answers were posted on the website Reddit.com hours before U.S. testing started.
Bob Schaeffer, Public Education Director of the National Center for Fair & Open Testing (FairTest), explained, “Early Friday evening, May 31, 2019, FairTest received a call from East Asia. The anonymous caller offered us an advance copy of the SAT Biology E/M Subject Test scheduled to be administered on Saturday, June 1. At 6:13pm EDT on Friday, May 31, we received an email copy of a complete Biology exam, apparently excerpted from the official booklet of SAT Subject Tests.”
Schaeffer continued, “Early Saturday morning U.S. time, hours before test administration began in the Western Hemisphere, Biology Test questions and answers were posted on a Reddit message board (https://www.reddit.com/r/Sat/comments/bvi735/official_june_2019_biology_discussion/). The content was identical to the material FairTest had received the previous evening. The Reddit discussion site is open to the public and draws heavy traffic from test-takers.”
FairTest received advance copies of SAT exams from sources in Asia four other times in recent years. For many other administrations, actual exam items were posted online before U.S. students entered test centers. Given this history, Schaeffer says the latest incident raises serious questions:
– How widely was the leaked June 1 Biology Subject Test circulated? In a world of instantaneous, global communication, if an exam is compromised anywhere, the items can be distributed everywhere.
– Which other Subject Tests were also compromised? The file FairTest received on Friday night included the introduction to the Chemistry Subject Test; content from that exam was also posted on Reddit before the test was administered in the U.S.
– Will scores from students taking these compromised tests be reported as valid?
– How can any admissions office trust the integrity of the June 1, 2019 Subject Test results?
– When will the College Board admit that its test security programs have repeatedly failed?
FairTest’s Schaeffer concluded, “Along with the ‘Varsity Blues’ scandal, the College Board’s security problems provide more evidence that SAT scores are not a reliable measure of readiness for college. That is one reason why more than 1,025 accredited, bachelor-degree granting colleges and universities now have test-optional admissions policies (http://www.fairtest.org/university/optional).
– – 3 0 – –
Diane Ravitch's blog 
FairTest must be simply giddy over this.
Same as it ever was
SAT : Shared Answers Test
AMEN.
I can’t wait for the College Board (and it’s “products”) to be revealed as the racist fraud that it is. Once the testing nightmare crumbles, public education will be able to return to it’s former glory days. MEGA……Make Education Great Again! I never thought I would congratulate criminals, but these hackers that are breaching testing companies and posting the tests are doing us a big favor.
I am curious about why you call the College Board a racist fraud. What is the basis of this claim?
And so there is one more reason to count college entrance testing a fraud. How many must there be before we will throw this industry in the logical scrap heap? Someday, perhaps, I will go into an antique store and find a scrap of one of these tests.
The Phoenix
The SAT’s a phoenix
That rises from the ash
So better get some Kleenex
The SAT WILL last
“How can any admissions office trust the integrity of the June 1, 2019 Subject Test results?
Integrity was never a part of the equation as these types standardized test results are COMPLETELY INVALID from the start. Wilson showed us those many invalidities involved in the standards and testing malpractice regime in his never refuted nor rebutted 1997 dissertation “Educational Standards and the Problem of Error” found at: https://epaa.asu.edu/ojs/article/viewFile/577/700
Brief outline of Wilson’s “Educational Standards and the Problem of Error” and some comments of mine. (updated 6/24/13 per Wilson email)
1. A description of a quality can only be partially quantified. Quantity is almost always a very small aspect of quality. It is illogical to judge/assess a whole category only by a part of the whole. The assessment is, by definition, lacking in the sense that “assessments are always of multidimensional qualities. To quantify them as unidimensional quantities (numbers or grades) is to perpetuate a fundamental logical error” (per Wilson). The teaching and learning process falls in the logical realm of aesthetics/qualities of human interactions. In attempting to quantify educational standards and standardized testing the descriptive information about said interactions is inadequate, insufficient and inferior to the point of invalidity and unacceptability.
A major epistemological mistake is that we attach, with great importance, the “score” of the student, not only onto the student but also, by extension, the teacher, school and district. Any description of a testing event is only a description of an interaction, that of the student and the testing device at a given time and place. The only correct logical thing that we can attempt to do is to describe that interaction (how accurately or not is a whole other story). That description cannot, by logical thought, be “assigned/attached” to the student as it cannot be a description of the student but the interaction. And this error is probably one of the most egregious “errors” that occur with standardized testing (and even the “grading” of students by a teacher).
Wilson identifies four “frames of reference” each with distinct assumptions (epistemological basis) about the assessment process from which the “assessor” views the interactions of the teaching and learning process: the Judge (think college professor who “knows” the students capabilities and grades them accordingly), the General Frame-think standardized testing that claims to have a “scientific” basis, the Specific Frame-think of learning by objective like computer based learning, getting a correct answer before moving on to the next screen, and the Responsive Frame-think of an apprenticeship in a trade or a medical residency program where the learner interacts with the “teacher” with constant feedback. Each category has its own sources of error and more error in the process is caused when the assessor confuses and conflates the categories.
Wilson elucidates the notion of “error”: “Error is predicated on a notion of perfection; to allocate error is to imply what is without error; to know error it is necessary to determine what is true. And what is true is determined by what we define as true, theoretically by the assumptions of our epistemology, practically by the events and non-events, the discourses and silences, the world of surfaces and their interactions and interpretations; in short, the practices that permeate the field. . . Error is the uncertainty dimension of the statement; error is the band within which chaos reigns, in which anything can happen. Error comprises all of those eventful circumstances which make the assessment statement less than perfectly precise, the measure less than perfectly accurate, the rank order less than perfectly stable, the standard and its measurement less than absolute, and the communication of its truth less than impeccable.”
In other words all the logical errors involved in the process render any conclusions invalid.
The test makers/psychometricians, through all sorts of mathematical machinations attempt to “prove” that these tests (based on standards) are valid-errorless or supposedly at least with minimal error [they aren’t]. Wilson turns the concept of validity on its head and focuses on just how invalid the machinations and the test and results are. He is an advocate for the test taker not the test maker. In doing so he identifies thirteen sources of “error”, any one of which renders the test making/giving/disseminating of results invalid. And a basic logical premise is that once something is shown to be invalid it is just that, invalid, and no amount of “fudging” by the psychometricians/test makers can alleviate that invalidity.
Having shown the invalidity, and therefore the unreliability, of the whole process Wilson concludes, rightly so, that any result/information gleaned from the process is “vain and illusory”. In other words start with an invalidity, end with an invalidity (except by sheer chance every once in a while, like a blind and anosmic squirrel who finds the occasional acorn, a result may be “true”) or to put in more mundane terms crap in-crap out.
And so what does this all mean? I’ll let Wilson have the second to last word: “So what does a test measure in our world? It measures what the person with the power to pay for the test says it measures. And the person who sets the test will name the test what the person who pays for the test wants the test to be named.”
In other words it attempts to measure “’something’ and we can specify some of the ‘errors’ in that ‘something’ but still don’t know [precisely] what the ‘something’ is.” The whole process harms many students as the social rewards for some are not available to others who “don’t make the grade (sic)” Should American public education have the function of sorting and separating students so that some may receive greater benefits than others, especially considering that the sorting and separating devices, educational standards and standardized testing, are so flawed not only in concept but in execution?
My answer is NO!!!!!
One final note with Wilson channeling Foucault and his concept of subjectivization:
“So the mark [grade/test score] becomes part of the story about yourself and with sufficient repetitions becomes true: true because those who know, those in authority, say it is true; true because the society in which you live legitimates this authority; true because your cultural habitus makes it difficult for you to perceive, conceive and integrate those aspects of your experience that contradict the story; true because in acting out your story, which now includes the mark and its meaning, the social truth that created it is confirmed; true because if your mark is high you are consistently rewarded, so that your voice becomes a voice of authority in the power-knowledge discourses that reproduce the structure that helped to produce you; true because if your mark is low your voice becomes muted and confirms your lower position in the social hierarchy; true finally because that success or failure confirms that mark that implicitly predicted the now self-evident consequences. And so the circle is complete.”
In other words students “internalize” what those “marks” (grades/test scores) mean, and since the vast majority of the students have not developed the mental skills to counteract what the “authorities” say, they accept as “natural and normal” that “story/description” of them. Although paradoxical in a sense, the “I’m an “A” student” is almost as harmful as “I’m an ‘F’ student” in hindering students becoming independent, critical and free thinkers. And having independent, critical and free thinkers is a threat to the current socio-economic structure of society.
“. . .the College Board’s security problems provide more evidence that SAT scores are not a reliable measure of readiness for college.”
Those score aren’t a measure of anything. Pray tell what is the standard unit of measurement used in the SAT. Hint: There isn’t any.
The most misleading concept/term in education is “measuring student achievement” or “measuring student learning”. The concept has been misleading educators into deluding themselves that the teaching and learning process can be analyzed/assessed using “scientific” methods which are actually pseudo-scientific at best and at worst a complete bastardization of rationo-logical thinking and language usage.
There never has been and never will be any “measuring” of the teaching and learning process and what each individual student learns in their schooling. There is and always has been assessing, evaluating, judging of what students learn but never a true “measuring” of it.
But, but, but, you’re trying to tell me that the supposedly august and venerable APA, AERA and/or the NCME have been wrong for more than the last 50 years, disseminating falsehoods and chimeras??
Who are you to question the authorities in testing???
Yes, they have been wrong and I (and many others, Wilson, Hoffman etc. . . ) question those authorities and challenge them (or any of you other advocates of the malpractices that are standards and testing) to answer to the following onto-epistemological analysis:
The TESTS MEASURE NOTHING, quite literally when you realize what is actually happening with them. Richard Phelps, a staunch standardized test proponent (he has written at least two books defending the standardized testing malpractices) in the introduction to “Correcting Fallacies About Educational and Psychological Testing” unwittingly lets the cat out of the bag with this statement:
“Physical tests, such as those conducted by engineers, can be standardized, of course [why of course of course], but in this volume , we focus on the measurement of latent (i.e., nonobservable) mental, and not physical, traits.” [my addition]
Notice how he is trying to assert by proximity that educational standardized testing and the testing done by engineers are basically the same, in other words a “truly scientific endeavor”. The same by proximity is not a good rhetorical/debating technique.
Since there is no agreement on a standard unit of learning, there is no exemplar of that standard unit and there is no measuring device calibrated against said non-existent standard unit, how is it possible to “measure the nonobservable”?
THE TESTS MEASURE NOTHING for how is it possible to “measure” the nonobservable with a non-existing measuring device that is not calibrated against a non-existing standard unit of learning?????
PURE LOGICAL INSANITY!
The basic fallacy of this is the confusing and conflating metrological (metrology is the scientific study of measurement) measuring and measuring that connotes assessing, evaluating and judging. The two meanings are not the same and confusing and conflating them is a very easy way to make it appear that standards and standardized testing are “scientific endeavors”-objective and not subjective like assessing, evaluating and judging.
That supposedly objective results are used to justify discrimination against many students for their life circumstances and inherent intellectual traits.
C’mon test supporters, have at the analysis, poke holes in it, tell me where I’m wrong!
I’m expecting that I’ll still be hearing the crickets and cicadas of tinnitus instead of reading any rebuttal or refutation.
So right! And when did Coleman’s College Board start giving subject tests in addition to the traditional SAT? That’s specific stupid added on to general stupid. Our students are being herded like poorly treated livestock.
I tell both of my kids that there is NO test that is able to determine how smart they are or how smart anyone else is. Even IQ tests can’t determine intelligence. Kids have been fed this lie by the school systems for so long that they refuse to believe the truth. The SAT, AP, ACT only show how well someone can take a test….and a very stupid and narrow test at that! I can’t wait to see the College Board and David Coleman fall like a house of cards in the wind.
The tests given in high school classes can, however, keep students from graduating high school, no matter how smart they are.
But, Teaching Economist, we’ve long known that high-school grades are better predictors of success in college than the SAT is.
Setting aside the obvious difference between tests designed by teachers that reflect the material they covered (as opposed to the SAT designed by Oxfordmorons like David Coleman at College Board to cover anything they damned well please), the many testS (plural) given in high school classes are not just ONE test (singular).
What a difference a single S can make.
Bob,
That is not relevant to my point here. With the notable exception of Duane, what I take to be the orthodox position here is that standardized tests are completely uninformative and the tests given in high school classes are fully informative. I do not think that is correct and I am especially concerned because it is the tests given in high school classes that have a much larger impact on students than standardized exams.
SDP,
I certainly agree that the “s” is important, but do remember that an exam or two in a course required for high school graduation could determine if a student graduates from high school or not.
The most academically oriented of my children took 14 standardized exams in high school: PSAT, SAT, SAT Math, SAT Physics, SAT Chemistry, and 9 AP exams. Is this a sufficient number of testS to be meaningful?
You are mistaken, Duane
There IS a standard unit of measurement for the Common Core aligned SAT: SCCAT (hat tip to Bob Shepherd for the acronym)
The higher your SAT score, the more SCCATs you get. That’s why schools like Harvard, Princeton and Yale are so full of SCCAT.
I just discovered that there is actually a test called the SCAT
“The School and College Ability Test (SCAT), is a standardized test conducted in the United States that measures math and verbal reasoning abilities in gifted children.”
https://en.m.wikipedia.org/wiki/School_and_College_Ability_Test
Ha ha ha ha.
The people who come up with this stuff are completely clueless.
Do you want to know how reliable the SCAT test really is? Both of my kids had to take it to get into GT classes at the ES level. Our school was a Title I school so the “score” to get into the classes was 65%> yet if you went to the rich school with very few FARMS kids, the score was 90%> and the school that had the largest population of wealthy Asian students (we are a VERY segregated school system) the score was 95%>. It took me a while to catch onto that farce but I know now!
For those of us who self-identify as suffereing from AIIDS* please help me with what the acronym SCCAT stands for. TIA!
*AIIDS = Acronym Identification Impairment Disorder Syndrome (to be listed in the DSM-X)
Duane
The CC stands for Common Core
But unfortunately, I can’t say what the S, A and T stand for because the College Board changes their claim about that every few years.
It has been changed so many times that i am inclined to say that SAT stands for nothing (just like the test)
And, of course, scat is self explanatory, especially when you step in it.
LisaM
Thanks for the info on the SCAT
I still can’t believe anyone would actually name a test that they considered serious ” SCAT”
Obviously they did not know what the word means.
“To determine whether you are gifted and talented , you must take this s**t test!”
Maybe “scat” is not an SAT word?
Regardless, it is an excellent word for a standardized test — and excellent word in general.
I especially like variants like “scatological.”
You know, before I found that wikipedia entry for SCAT, I would have guessed that such a test was used in wildlife biology to identify scat of various animals in the woods.
That would have made sense.
I actually have a Field guide to Animal Tracks (by Olaus Juris) that has lots of drawings of scat in it.
Unlike the standardized test with its namesake, scat is very useful
Olaus Murie — not Juris
Piece of scat self-correct!
See, a useful word.
Kudos to FairTest for being a clearinghouse, chronicling all the missteps and atrocious misuses we have suffered at the hands of testing over the decades. It has played a vital role in combatting the evils of faux quantification.
Bob Schaeffer and Monty Neill kept FairTest alive. We were all young at one time in many ways. We testified in favor of New York State’s successful good government legislation circa 1980 that applied to college admissions exams (SAT) and required test developers (ETS) to disclose research data bearing on test development and impact. It was popularly known as the Truth-in-Testing law and for a while it slowed down the industry that gave us college entrance via verbal and numerical test score.
Since that time, entropy has set in, and what was once a great idea has become an oxymoron. Hoped for reforms were limited in application and short-lived. Now we see how testing calls the tune in all grades leading up to high school graduation. I hope the fact that colleges have now come to discard SAT and ACT scores in considering worthy applicants had something to do with the skepticism that was inherent in T-in-T. And I’d like to see the public and politicians revive that concept and bring it to forceful realization regarding all commercially-produced exams
Anyone who has taught high-school kids knows that they are extremely emotionally unstable. It’s a difficult time. It’s the time in which we all struggle with establishing an identity that will be acceptable to/accepted by the others around us. One way in which kids do that is by rebelling against their parents and teachers and older authorities in general. This rebellion can take forms both positive and negative. On the positive side, many turn to resistance against how older people have messed things up for them–have given them human-caused climate change or dying oceans or Trump and his stupid wall. On the negative side, many turn to destructive behaviors of which older people disapprove–drinking and drugs and petty theft (shoplifting) and dangerous sexual experimentation for which they are not ready physically or emotionally. High-school kids tend to be extreme about everything–extremely idealistic and extremely inclined to go further, in their beliefs about the world, than their actual knowledge and experience rationally allow. They are sensitive and volatile and more than a little bit crazy, like caged tigers.
For a long time, great teachers in the humanities (English, history, art, theatre, music, languages) and in the sciences approached as a humane undertaking were able to harness that youthful idealism, that desire to define themselves as change agents over and against the adult world. In every classroom, there is the overt curriculum and then there are the hidden curricula that get taught incidentally. An extremely important part of the hidden curriculum in those classes in high school was always that a great teacher would use great cultural products from the past to harness that idealism and desire for an identity: “I am a writer, a musician, a linguist, a historian, a biologist, in the making,” the student would learn to say of him or herself. “I am Yolanda the poet.” An English class in which the overt curriculum as, say, study of Slaughterhouse Five, would become one in which, because the class was focused on what authors had to say, the hidden curriculum taught that people do (and rationalize to themselves) really stupid and evil things in war. And the kids would get all fired up about that. One in which the overt curriculum as American literature of the Puritan Era would become one in which the hidden curriculum taught Puritan values like individualism and local government and rebellion against tyranny and the horrors that can occur when people don’t practice acceptance and toleration (e.g., the genocide against the indigenous population in the Americas). And because kids were getting something from it–a sense of their own identity or a purpose or cause to be fired up about, they would learn that learning itself was of value. And what would last and be important from that high-school experience–what would not, perhaps, bear its fruit for years but would, indeed, bear fruit, would be that learning.
Not so now. English class has become all about applying item x from the Gates/Coleman bullet list to text snippet y in preparation for the ALL IMPORTANT test that will determine whether the kid will be acceptable for advancement. Kids have been robbed, by Ed Deform, by this testing mania, of humane education, of the hidden curriculum that taught them, most importantly, to become intrinsically motivated, life-long learners. No one ever got fired up by a set of test prep exercises.
We have an epidemic, now, in the US of high-school kids who are extraordinarily stressed out, who don’t see a future for themselves, who cut themselves and suffer from depression and anorexia, who commit suicide. If you teach in a high-school, you see this all the time, but especially at the end of the year, as testing season approaches. The kids, having been herded and cajoled and threatened all year; having spent a year sitting in class for an hour, getting up and moving for three minutes, sitting in another, and doing this six or seven times a day; now face the very real prospect of failure on invalid, capricious standardized tests, and they are stressed, stressed, stressed and ANGRY. The testing is AN ACT OF VIOLENCE AGAINST CHILDREN.
An entire generation of students has now been subjected to the standards-and-testing regime. And the results are in. We now KNOW that it has fulfilled NONE of its promises. It hasn’t improved learning outcomes. It hasn’t closed achievement gaps. But it has narrowed and distorted curricula and pedagogy and made our children SICK.
Enough. Standardized testing is a vampire that sucks the lifeblood out of education. Put a stake in it.
One minor correction:
Testing is a thing, not a person. You don’t put a stake in the Vampire’s behavior, you put a stake in the Vampire.
I am speaking metaphorically of course, but the behavior ,(testing) is going to persist until the Vampire’s behind it are driven out, blacklisted and otherwise banished from education.
“The VAMpire States of America”
The VAMpire States
Are really great
Cuz VAMpires run the ed
They love the VAMs
And other shams
A Night of Living Dead
My love is a red, red rose.
One minor correction: A love is not a plant. And you can’t make him or her into rose hips tea.
But yes, when the correction occurs, and the ridiculous testing is ended, I hope that people will remember the Vichy collaborators among administrators, politicians, state department officials, and, especially, EduPundits who subjected our children to this abuse for so many years.
My vampire has a red red nose
Red lips too.
And a red chin.
I can’t wait to see how the crackpot (fake) statisticians at College Board deal with this one.
I am sure they will come up with some way of claiming they can adjust for it (like they fudge all their other stuff)
Psychometricians are better fudge makers even then economists.