This is a fascinating article.
Mimi Swartz of the Texas Monthly asks an important question: Are Texas kids failing or are the tests rigged against them?
Researchers with no axe to grind say the state tests are two grade levels above where the kids are. The state doesn’t agree.
State Commissioner of Education Mike Morath is not an educator, though he was a school board member in Dallas. He was appointed by rightwing Governor Gregg Abbott, a leader in the effort to defund and privatize public schools. Of course, he believes that public schools are horrible and charters are wonderful. He will believe anything that puts public schools in a bad light, regardless of evidence or research. The legislature slashed the state budget by over $5 billion in 2011, and has never restored funding to where it was before the 2008 recession.
Swartz begins:
“Over the last few years, something strange has been happening in Texas classrooms. Accomplished teachers who knew their kids were reading on grade level by virtually all other measures were seeing those same kids fail the STAAR, the infamous State of Texas Assessments of Academic Readiness test.
“The effect on students was predictable: kids who were diligently doing their homework and making good grades in class were suddenly told they were failing in the eyes of the state, which wasn’t so great for their motivation. Parents were desperate to find out why their once high-performing kids were suddenly seen as stumbling. Teachers felt like failures, too, but had no idea what they were doing wrong, after years of striving to adopt practices proven in successful schools across the country. What’s more, the test results were quickly weaponized by critics of Texas public schools, many of whom advocate state-funded vouchers that would allow parents to send their kids to religious and other private schools.
“The stakes of such exams are perilously high. The STAAR test, developed by the Educational Testing Service in Princeton, N.J., had replaced one provided by the British firm Pearson, which Texas officials considered too easy. The STAAR test is used to evaluate students, teachers, individual schools and principals, school districts, and, by extension, the entire enterprise of public education in Texas. Fifth and eighth graders who fail the test can be forced to repeat a grade; high school students may not graduate if they don’t pass three of the five STAAR year-end exams.
“On its face, this approach makes sense. This is, after all, the Age of Accountability, and, according to Governor Greg Abbott and other prominent state leaders, only 40 percent of Texas third graders are reading at grade level. The STAAR numbers are cited as positive proof of that. Texas has to get its kids and its public schools up to the highest standards if we want to have the educated workers and informed citizens we need. There isn’t a minute to lose.
“This reasoning may explain why a report issued in 2012 by two associate professors at Texas A&M was overlooked. Called “STAAR Reading Passages: The Readability is Too High,” by Susan Szabo and Becky Sinclair, the report suggested that questions on the STAAR test were too hard to accurately measure whether students were reading at their grade level.
“The researchers’ examination of five different “readability tests”—commonly used academic measures that rate the appropriateness of written passages for various grade levels—showed, for instance, that in order to comprehend various passages, a third grader would have to read on a fifth-grade level. A fifth grader would have to read on a seventh-grade level, and so on. Generally, the testing showed a gap of about two years. Szabo and Sinclair’s paper made no waves. The STAAR test was new, and if there was a warning included in the research, no one in power thought to consider it. An organization called Texans Advocating for Meaningful Student Assessment lodged protests, but they were rebuffed.
“Years passed. The STAAR reading test reported more failures and stirred more concerns. Teachers and administrators continued to see that the STAAR scores didn’t “align” with other indicators of reading levels. Specifically, the numbers didn’t match those of the Lexile scale, which is regarded nationally as the standard gauge of any publication’s degree of difficulty. (Libraries use the Lexile scale to direct kids to age-appropriate books.)
“In 2016, another study was released, this time by Michael Lopez and Jodi Pilgrim, two professors at the University of Mary Hardin-Baylor, in Belton, Texas. They, too, found that readability formulas showed that the STAAR test contained too many difficult passages for the targeted age groups—“materials may be problematic for teaching and learning”—which confirmed what many teachers were seeing in their classrooms. That same year, a group of fifty Texas school superintendents lodged their protests with the Texas Education Agency (TEA), which administers the STAAR test.
“It’s easy, especially in Texas, to explain away some of the complaints as just so much whining. According to recent Education Week studies, our state ranks 40th in education quality. The blame for our sad showing has been placed on allegedly unqualified and unaccountable teachers, uninvolved parents, and corrupt administrators and school boards.
“But what if that showing isn’t as sad as we’ve been told? What if the STAAR test isn’t measuring what it says it’s measuring: i.e., that a third grader is reading at a third-grade level, rather than a fifth-grade level?….
“Morath did not respond to our request for an interview, but we were able to speak with Jeff Cottrill, TEA’s deputy commissioner of standards and engagement. He explained that TEA’s research on the STAAR reading test included early reviews by Texas teachers and students. “The test is rooted in Texas standards and reviewed by Texas teachers and field tested by Texas students,” Cottrill said. “I have to tell you the process by which TEA determines what goes in this test is solid.” Critics dismiss that method as nothing more than “a gut check,” as none of the test passages were run through standard readability measurements such as the Lexile. Cottrill confirmed that the test was not sent through a Lexile analysis. “TEA relies much more on people to assess the quality of the test than computer based algorithms… Some Dr. Seuss books are actually written at a higher Lexile than The Grapes of Wrath,” he said.
“The Lexile scale was not the only readability test by which researchers outside the TEA have evaluated the STAAR reading test. Dee Carney, the Austin testing expert, pointed out that the A&M research used five readability studies and the Mary Hardin-Baylor research used six. Chambers [the superintendent of the Alief district and president of the Texas School Alliance, representing the state’s largest districts] says new research conducted at A&M is to be released in the next few months and shows even more misalignment, or failing kids, today than in 2012. “If the decision was made to test kids in reading passages that are above their grade level, everyone needs to know that,” Chambers said. “If a third grade reading test is meant to determine if a student is reading at the third grade level, then the test questions should be based solely on what was taught in [and before] third grade, not what might be taught in the fourth, fifth, sixth, or seventh grade.”
“The consequences, Chambers said, can be severe. “To me, here is the bottom line: if Texas expects every third grader to read like a fifth grader or every fourth grader to read like a sixth grader, then we all need to be prepared to see lower performance. Based on all the expert information that has been provided, these unrealistic standards have the potential to destroy learning.”
Question from me: can anyone name a book by Dr. Seuss that has a higher Lexile level than “Grapes of Wrath”?
Proposal: How aboutif Governor Abbott, Lt. Governor Dan Patrick, and Superintendent Morath agree to take the tests in English and math and publish their scores? 12th grade? Eighth grade? How about it, guys?
The tests based on the CCSS are rigged against students in many states, not just Texas. Regardless of the readability scale used, the tests are written on the frustration level of students, and these tests are not providing accurate information on students’ reading comprehension. The objective of the test is not to provide meaningful information on students. The main objective is to provide a wide net of “failing schools” eligible for takeover and privatization. To make matters worse, Texas threatens students with repeating the grade if a student’s parent decides to opt students out. Politicians are working with privatizers to inflict a punitive policy on students in order to enforce their mandate.
Although there is no discussion about the math test in this article, there are reported problems about them as well. Questions are often vague, and students are unsure as what is being asked. Likewise, only one solution to problems is being accepted as the “correct” answer even though there is more than one way to solve a problem.
My grandson’s school is nervous about the STARR test as the school has many poor and language minority students in it. The administrators and teachers know what is at stake, and they are worried.
AND….just because a kid in 3rd grade can read on a 5th grade level DOES NOT mean that they are able to comprehend on a 5th grade level. Reading and comprehension go hand in hand.
It is fascinating. I’m surprised they had the courage to even question the tests, because the response is predictable- “making excuses!”
One must blindly accept whatever test is produced, or be smeared as lazy and unmotivated.
They bumped up the difficulty of the tests in Ohio and now the entire public must pretend that every public school dropped two letter grades because those schools are markedly worse, when all of us with kids in public schools know full well that they bumped up the difficulty of the tests.
Why is this so hard? If ed reformers want more difficult tests who not just say “we bumped up the tests so more of you are failing”. Why play these stupid games like this is “science”? We’re really not that stupid. We noticed that they changed the tests. Why are the insisting we compare apples and oranges, when literally everyone knows that’s what’s happening?
Just say “we decided there’s a higher bar” and stop smearing students and teachers and schools with this fantasy that these numbers are some kind of objective standard when they CHANGE THE MEASURE every time enough ed reformers lobby to have them changed. It’s not even fair to students. Tell them the truth.
Maybe we should just declare these manipulative tests invalid. Why are third graders expected to read at a fifth grade level in order to be declared proficient? It is significantly above the range of grade level expectation for wholly POLITICAL purposes. It’s rigged!
I think this is the same test my daughter has to take as a part of a gifted program. While she is in a great program, with great teaching in her area of giftedness, if that is a word, I have never put much stock in the scores on the tests. I have repeatedly observed that her ability to read something and understand it relates to whether she has prior knowledge of something more often than it relates to vocabulary or complexity of sentence structure. The Starr always throws her high on the reading level scale, but she still will come to me with basic questions about content.
Without much experience, therefore, I tend to be skeptical, especially in light of the use of testing to stack children as a part of their education. How does this help them?
Due to the lobbying of ed reformers and due to Ohio state government’s complete and utter capture by that lobby, Ohio actually changed the measurement system to benefit charters. The whole analysis was based on charters. They needed to bump up charter scores so they took us all to “value added”.
Now we’re in the fantasy part of this sequence, where we all pretend they’re using the same measure as 2 years ago.
They skew the data at the outset. The process itself is captured and geared to lower public school grades and offer charters a safe harbor.
No parent should rely on it. It’s meaningless.
It’s just so manipulative. They’re incapable of presenting a straightforward argument to defend their ideas so they play games with these people.
What’s the worse thing that would happen if they told the truth? If they told Texas public schools “we’re upping the cut score”. Then they wouldn’t be able to smear these people because they’d have to admit they raised the bar? Their political campaigns against public schools would be less effective?
They’re not partners with these schools. They’re not even straight with them. They don’t even show them that level of respect. They now have the ridiculous result where the schools have to pretend these comparisons are valid, so that ed reformers can use the comparisons to put in policy they wanted anyway. It’s one big lie.
To answer the question: Yes, of course they are.
All standardized tests are “rigged”. Noel Wilson has shown us in his never refuted/nor rebutted 1997 dissertation all of the onto-epistemological errors and falsehoods and psychometric fudges that render the tests completely invalid. Any usage of the results can ONLY BE “vain and illusory”. . . in other words horse manure. The standards and testing malpractice regime violates the very being of students and harms all of them. To more fully understand what those error, falsehoods and psychometric fudges are one should read and comprehend Wilson’s “Educational Standards and the Problem of Error”. https://epaa.asu.edu/ojs/article/viewFile/577/700
Brief outline of Wilson’sand some comments of mine:
A description of a quality can only be partially quantified. Quantity is almost always a very small aspect of quality. It is illogical to judge/assess a whole category only by a part of the whole. The assessment is, by definition, lacking in the sense that “assessments are always of multidimensional qualities. To quantify them as unidimensional quantities (numbers or grades) is to perpetuate a fundamental logical error” (per Wilson). The teaching and learning process falls in the logical realm of aesthetics/qualities of human interactions. In attempting to quantify educational standards and standardized testing the descriptive information about said interactions is inadequate, insufficient and inferior to the point of invalidity and unacceptability.
A major epistemological mistake is that we attach, with great importance, the “score” of the student, not only onto the student but also, by extension, the teacher, school and district. Any description of a testing event is only a description of an interaction, that of the student and the testing device at a given time and place. The only correct logical thing that we can attempt to do is to describe that interaction (how accurately or not is a whole other story). That description cannot, by logical thought, be “assigned/attached” to the student as it cannot be a description of the student but the interaction. And this error is probably one of the most egregious “errors” that occur with standardized testing (and even the “grading” of students by a teacher).
Wilson identifies four “frames of reference” each with distinct assumptions (epistemological basis) about the assessment process from which the “assessor” views the interactions of the teaching and learning process: the Judge (think college professor who “knows” the students capabilities and grades them accordingly), the General Frame-think standardized testing that claims to have a “scientific” basis, the Specific Frame-think of learning by objective like computer based learning, getting a correct answer before moving on to the next screen, and the Responsive Frame-think of an apprenticeship in a trade or a medical residency program where the learner interacts with the “teacher” with constant feedback. Each category has its own sources of error and more error in the process is caused when the assessor confuses and conflates the categories.
Wilson elucidates the notion of “error”: “Error is predicated on a notion of perfection; to allocate error is to imply what is without error; to know error it is necessary to determine what is true. And what is true is determined by what we define as true, theoretically by the assumptions of our epistemology, practically by the events and non-events, the discourses and silences, the world of surfaces and their interactions and interpretations; in short, the practices that permeate the field. . . Error is the uncertainty dimension of the statement; error is the band within which chaos reigns, in which anything can happen. Error comprises all of those eventful circumstances which make the assessment statement less than perfectly precise, the measure less than perfectly accurate, the rank order less than perfectly stable, the standard and its measurement less than absolute, and the communication of its truth less than impeccable.”
In other words all the logical errors involved in the process render any conclusions invalid.
The test makers/psychometricians, through all sorts of mathematical machinations attempt to “prove” that these tests (based on standards) are valid-errorless or supposedly at least with minimal error [they aren’t]. Wilson turns the concept of validity on its head and focuses on just how invalid the machinations and the test and results are. He is an advocate for the test taker not the test maker. In doing so he identifies thirteen sources of “error”, any one of which renders the test making/giving/disseminating of results invalid. And a basic logical premise is that once something is shown to be invalid it is just that, invalid, and no amount of “fudging” by the psychometricians/test makers can alleviate that invalidity.
Having shown the invalidity, and therefore the unreliability, of the whole process Wilson concludes, rightly so, that any result/information gleaned from the process is “vain and illusory”. In other words start with an invalidity, end with an invalidity (except by sheer chance every once in a while, like a blind and anosmic squirrel who finds the occasional acorn, a result may be “true”) or to put in more mundane terms crap in-crap out.
And so what does this all mean? I’ll let Wilson have the second to last word: “So what does a test measure in our world? It measures what the person with the power to pay for the test says it measures. And the person who sets the test will name the test what the person who pays for the test wants the test to be named.”
In other words it attempts to measure “’something’ and we can specify some of the ‘errors’ in that ‘something’ but still don’t know [precisely] what the ‘something’ is.” The whole process harms many students as the social rewards for some are not available to others who “don’t make the grade (sic)” Should American public education have the function of sorting and separating students so that some may receive greater benefits than others, especially considering that the sorting and separating devices, educational standards and standardized testing, are so flawed not only in concept but in execution?
My answer is NO!!!!!
One final note with Wilson channeling Foucault and his concept of subjectivization:
“So the mark [grade/test score] becomes part of the story about yourself and with sufficient repetitions becomes true: true because those who know, those in authority, say it is true; true because the society in which you live legitimates this authority; true because your cultural habitus makes it difficult for you to perceive, conceive and integrate those aspects of your experience that contradict the story; true because in acting out your story, which now includes the mark and its meaning, the social truth that created it is confirmed; true because if your mark is high you are consistently rewarded, so that your voice becomes a voice of authority in the power-knowledge discourses that reproduce the structure that helped to produce you; true because if your mark is low your voice becomes muted and confirms your lower position in the social hierarchy; true finally because that success or failure confirms that mark that implicitly predicted the now self-evident consequences. And so the circle is complete.”
In other words students “internalize” what those “marks” (grades/test scores) mean, and since the vast majority of the students have not developed the mental skills to counteract what the “authorities” say, they accept as “natural and normal” that “story/description” of them. Although paradoxical in a sense, the “I’m an “A” student” is almost as harmful as “I’m an ‘F’ student” in hindering students becoming independent, critical and free thinkers. And having independent, critical and free thinkers is a threat to the current socio-economic structure of society.
As “vain and illusory” testing results may be, privatizers are using test results to destroy public schools and undermine students’ self esteem. These tests are tools of a political agenda and serve no academic purpose.
This is but one of the ways tests are rigged. Another is that even so-called “criterion-referenced” tests are still designed to show a bell-curve distribution, so there will always be kids who “fail”. If all kids “pass”, then they just make the tests “harder” to get back to a “proper” bell curve. Even if we got to a point where every 3rd grader could pass a test like the GRE, they’d still have to make it “harder” so that some kids would “fail”. It’s a losing game.
If you want more reasons why the tests are rigged, I’m sure someone can quote some Wilson….
BTW, in case it’s not obvious, this rigging is by no means confined to Texas.
For God so loved the world that he sent his only son with a bell shaped curve to make sure someone was always a loser. Hmmm I think I misquoted that verse.
There is no surprise in this finding. The tests, as Diane reported from day one, had as its only purpose to demonize the teachers and the schools so that they would fail and be replaced by private charters. We are seeing the result of the NCLB act, which was the clever ploy that the POWER-ELITE used to foist disinformation n the media which THEY OWN.
It is the turn of Texas to be turned into privatized schools… to demonstrate with bogus evaluations (i.e tests) . how incompetent the ‘loser teachers’ are — a pretense they used across the 15,880 SEPARATE SCHOOLS SYSTEMS IN 50 STATES– to throw the genuine dedicated, educated, and EXPERIENCED teacher-practiitioners out of the schools!
Look at LA and NYC to see how it began.. It worked so well in the 2 largest school systems that they convinced legislators across the nation to take over the schools, and they told the people, in non-stop disinformation campaigns, how those ‘bad’ teachers (i.e experienced professionals) needed to GO!
I used to call these disruptors (who Diane labels the ‘billionaire’s boys club”) oligarchs, plutocrats, or ‘the cabal, but now I think that the appellation ‘power elite,’ is most appropriate, since I began reading Gary Brumback’s, “Life’s Triangles and America’s Power Elites”;(2019). I think this book offers us the CONTEXT for what is afoot on our landscape now, where the marauders who are devastating the middle class — know that they need GET THE CITIZENS as children, so as to end all prior knowledge of history, and promote ignorance.
As one review points out, Brumback offers a “causal relationships of greed, power, hierarchy, elitism, pathology, going back to this country’s earliest years and now advancing into today’s predatory capitalism and parasitic economics”
To define the ‘Power Elite, he “uses a variety of wide angle and telephoto angles in order to look deeper at the simple equation of the rich — with military might behind them — controlling the destiny of the country – us, its inhabitants – and the insecurity of the planet, from all the other inhabitants of 192 countries.”
The Power elite has given Texas the tests so they can show
Then Gates, and Walton and the Koch Brothers ( and the Educational Industrial Complex https://greatschoolwars.files.wordpress.com/2015/10/eic-oct_11.pdf
can write the curricula that dumbs-down the next generation of voters.
I would only add the word SADLY at the very beginning of your response: SADLY, there really is no surprise here.
sorry: Correction of the last paragraph
The Power elite has given Texas the tests so they can show the schools are failing, and the incompetent ‘loser teachers’ must be replaces.
Then, Gates, and Walton and the Koch Brothers ( and the Educational Industrial Complex https://greatschoolwars.files.wordpress.com/2015/10/eic-oct_11.pdf
can write the curricula that dumbs-down the next generation of voters.
How to Rig a Test Score
Adopt vague and subjective standards
Adopt performance (skill based) standards
Adopt developmentally inappropriate standards
Write vague and subjective tests using objective items
Write developmentally inappropriate tests
Write unreasonable scoring rubrics for ER items
Set cut scores to produce hyper-failure rates
Create a testing schedule that produces test fatigue
Implement no-stakes testing for students
Use an annual testing regime that cannot parse grade level differences
Test all ELL and all IEP students
Convince the general public that none of the above is happening
All of the rigging is intended to create “failing public schools” so the politicians and elites can send mostly poor minority students to fake public schools paid for by public money. Most likely these fake schools with fake teachers like TFA will get no better results, and perhaps much worse results. Now that’s a “vain illusion” that has never been proven to have any value at all! This is the billionaire dystopia for other people’s children.
I wonder what Trump would score on those tests — 4th grade maybe (probably a bit high)? Since he hates to read and brags about not liking to read, his literacy level must be very low. The more one reads, the higher their literacy levels.
Trump would be a good test case. Except that the results of these tests are pretty random if they are studied very much. It has been noted here often that such tests are more closely correlated with income level than anything else. Trump might do OK, or he might look horrible. I suspect that he would do very well if he had the prep classes like so many have. Seems to me that these tests might indicate something having to do with a person’s ability to take these tests, which is an indication of how hard they have worked at taking these tests. now there is a skill that will carry us deep into this century.
It wouldn’t be fair to let Trump study. One day a firing squad walks into the Oval Office, plunks the test down on whatever desk the Orange Idiot is using, and starts to a stopwatch.
He doesn’t have a choice. He takes the test or else …
This is serious . Can we leave President Trump out of the discussion, please?
No, we must never leave Trump out of discussion.
We must never forget what a slimeball he is.
We must never stick our heads in the sand and pretend this Orange Slime Ball doesn’t exist. Avoiding Trump is not going to make him go away.
He is still in the White House eating fast food, watching Fox news and Tweeting a frenzy of tweets full of lies, hate, and insults.
Trump is still holding fascist rallies full of hate language that recently caused one of his MAGA hat loving fanatics to attack a BBC cameraman at one of his rallies.
In addition, we must dig deeper to discover what’s going on in Trump’s administration to destroy and dismantle the U.S. government and the U.S. Constitution.
For instance, the Mar-a-Lago Trio who are now “illegally” running the VA without the approval of the US Congress. The goal of the Mar-a-Lago Trio is to dismantle the VA and destroy its efficiency so it will be easier to privatize the VA medical system.
“A new report questions how the Veterans Affairs Department is being run in the Trump administration. ProPublica says the VA has three ‘shadow rulers,’ three men who have never served in the U.S. military or government but have outsize influence over all department decisions. Nick Schifrin talks with Isaac Arnsdorf of ProPublica and Melissa Bryant of Iraq and Afghanistan Veterans of America.”
Educators and child psychologists cannot possibly know what children are capable of at any age level. this must be left to the politicians. Pass the legislation makes if possible to keep raising the bar for children. Just have children study quantum mechanics in first grade. Any GOOD teacher should be able to do that, especially with 40 in a classroom.