For many years, I was a staunch advocate of standardized testing. But I lost my enthusiasm for standardized testing after spending seven years on the governing board of NAEP (the National Assessment of Educational Progress). NAEP is the federal test administered every two years to measure academic progress in reading and math, as well as testing other subjects. The test takers are randomly selected; not every student answers the questions on any test. There are no stakes attached to NAEP scores for any student, teacher, or school. The scores are reported nationally and by state and for nearly two dozen urban districts. NAEP is useful for gauging trends.
Why did I lose faith in the value of standardized testing?
First, over the course of my term, I saw questions that had more than one right answer. A thoughtful student might easily select the “wrong” answer. I also saw questions where the “right” answer was wrong.
Second, it troubled me that test scores were so highly correlated with socioeconomic status. Invariably, the students from families with the highest income had the highest scores. Those from the poorest families had the lowest scores.
Third, the latter observation spurred me to look at this correlation between family wealth and test scores. I saw it on the results of every standardized test, be it the SAT, the ACT, or international tests. I wondered why we were spending so much money to tell us what we already knew: rich kids have better medical care, fewer absences, better nutrition, more secure and stable housing, and are less likely to be exposed to vermin, violence, and other health hazards.
Fourth, when I read books like Daniel Koretz’s “Measuring Up” and “The Testing Charade” and Todd Farley’s “Making the Grades: My Misadventures in the Standardized Testing Industry,” my faith in the tests dissipated to the vanishing point.
Fifth, when I realized that the results of the tests are not available until the late summer or fall when the student has a new teacher, and that the tests offer no diagnostic information because the questions and answers are top-secret, I concluded that the tests had no value. They were akin to a medical test whose result is available four months after you see the doctor, and whose result is a rating comparing you to others but utterly lacking in diagnostic information about what needs medication.
So, all of this is background to presenting a recent study that you might find useful in assessing the value of standardized tests:
Jamil Maroun and Christopher Tienken have written a paper that will help you understand why standardized tested is fatally flawed. The paper is on the web and its title is:
The Pernicious Predictability of State-Mandated Tests of Academic Achievement in the United States
Here is the abstract:
The purpose of this study was to determine the predictiveness of community and family demographic variables related to the development of student academic background knowledge on the percentage of students who pass a state-mandated, commercially prepared, standardized Algebra 1 test in the state of New Jersey, USA. This explanatory, cross-sectional study utilized quantitative methods through hierarchical regression analysis. The results suggest that family demographic variables found in the United States Census data related to the development of student academic background knowledge predicted 75 percent of schools in which students achieved a passing score on a state standardized high school assessment of Algebra 1. We can conclude that construct-irrelevant variance, influenced in part by student background knowledge, can be used to predict standardized test results. The results call into question the use of standardized tests as tools for policy makers and educational leaders to accurately judge student learning or school quality.
The paper was peer-reviewed. It was published last week.
Diane Ravitch's blog 
Now, if only the policy makers could read . . .
Such testing could be used as a guide for curriculum makers, but I have always wondered why such tests were used as indicators of what kids were learning or being taught. The fourth grade tests from one cycle were compared to the fourth grade tests of anther. But the two groups of kids are entirely different, so whether things got better or worse is due to . . . what, different cohorts?
That kids did better in one region might mean a better curriculum or delivery system, but after you factor out the effects mentioned above, what’s left?
LikeLike
Yes, and in this case it’s even worse as students are from different schools across the country which have different teachers, curricula and a whole range of other factors. With that many variables, how can one test say anything about anything really?? Even if you want to look at the results of a test of one teacher’s students over time, it’s still difficult because the students are always changing, the way the teacher teaches is always changing. Everything is always changing. How can one test say anything about all of these factors? It’s unfortunate that we don’t pay more attention to the growth of the student. When I was in 10th grade, I was failing Geometry miserably. My parents sat down with me every night for hours and helped me. My father was a doctor and my mother was a lawyer – very educated people. I ended up getting an 86 on the Geometry Regents Exam. It was as if I climbed Mount Everest. I’m sure other students didn’t put in the time I did and got higher marks. But for me it was a triumph of a huge magnitude. In the end it taught me more than Geometry. It taught me persistence and to face my fears. Our educational system takes none of this into account, and that’s pretty sad. I’ve had students who had so much trouble with French but I cut out all the learning French and drilled into them what I knew they would need to pass the test. I remember one student who got in the 90’s on the French regents but had such a hard time learning it. He didn’t have to know it. I taught him the keys to taking the test. So, in the end, what does it all mean????
LikeLike
So right, Mamie. There is so much the tests don’t take into account.
I have never forgotten when NYC adopted the VAM model for evaluating teachers. The best teachers got big increases in student scores.
Teachers of gifted students got bad grades because their students were already at the top.
The city’s “worst” teacher taught newcomers: some knew some English, others didn’t. They cycled in and out all year. The scores reflected them, not her teaching.
LikeLike
And what did my quitting teaching this kid French and drilling him with what he needed to pass the test say to him? I was basically telling him that learning French is irrelevant and unimportant. Passing this test is the important thing. Education is the means to an end and do what you can to reach that end. That’s what matters. And in my case with geometry, my 86 should have been a celebration in the eyes of our educational system. And it was to my parents and me.
LikeLike
Google Campbell’s Law.
LikeLike
Steve,
The media and certain think tanks spread the crazy belief that test scores in every grade should go up every year. Why should fourth graders be smarter this year that fourth graders two years ago?
LikeLike
I though I made that point. It is incoherent, yet our policy makers buy into it, which to me is always a sign of having a hidden agenda, maybe to destruction of “government schools”?
LikeLike
And a great point it is. Diane has made a point of how insane this is in her books, again and again, for years now. Here was my take on it way back in 2018:
LikeLike
The only thing necessary for the triumph
of test scores, is for the test GIVERS to
keep giving them.
After you factor in the impact of all the
“know-that” related to test scores, the
shortest distance between the kids and
testing is STILL the test givers.
Tests don’t harm kids UNTIL they
are given…
LikeLike
You’re more kind than me in giving the teachers and adminimals grief about giving the tests. GAGA Good German types they are. . . which sadly is about 99% of all of them.
LikeLike
Yes. Many of our schools were doing very well, thank you–until Reagan and the wrecking-crew Right got hold of education. Some “reformers” were well-intentioned, but some were out to weaken “liberal” education, teacher unions, and the rise of the working classes. I was there, as a teacher and teacher-union officer. The right-wing Republicans (not all Republicans) were always out to stymie or reverse “liberal” education. Sadly, some of the Clinton-Obama Democrats bought into the testing mania.
As to the standardized testing itself, we were taught at Ohio State that such tests were almost worthless–that essays and such were the best approaches to finding out what people have learned. Think of it: Would you teach quarterbacks how to throw a football with a series of statements and a standardized test? Of course not. And so you cannot teach democracy that way either. Schools have to “do” democracy if children are to learn to behave democratically.
Thanks for this accurate and honest update on current thinking about mandated standardized tests. I’m retired, but I hope future kids will be spared the miseducation of the testing mania.
LikeLike
As a practitioner, it has been rare for me to find scholarly writing that supports standardized testing as an effective tool to evaluate a school’s progress. Yet, while a principal, my states and districts doubled down on “high stakes testing” even after such analysis was provided. In the summer or fall of the year following a particular test, school staff faced all sorts of graphs from bar to scattershot all designed to justify the conclusions of policy makers that led to particular one size fits all instructional practices. Diane writes of experiences similar to mine where testing instruments were often flawed and the data was reported as autopsy rather than a diagnostic tool useful to teachers. Many in the realm of education have pointed out the imperfection of standardized tests, yet the press continued to report the scores as if they were infallible while policy makers misused the published results to justify defunding public schools, adding charters, implementing vouchers, adding technology, or all of the above. Data we got from studies such as the one cited in this blog was important, but the wordy conclusions were typically ignored. Standardized testing has proved it is an ineffective tool for student academic growth when all measures have demonstrated almost no change in student performance over the decades, whether from good or bad assessments. When high stakes testing began in North Carolina, all educators had to do was look at the poverty levels at schools to predict their results. It was that simple. To counter this narrative, the state adopted a secret algorithm that supposedly showed a schools growth which was then hailed as progress even though the same number of students from specific demographics continued to score below proficiency. As the state rewarded teachers for growth, they failed to increase funding to help the students in underprivileged communities succeed. Anyone who has participated in the standardized testing charade, whether principal, teacher, or parent, knows it is a fraud. The challenge is to get high stakes advocates to stop the practice.
LikeLike
Paul,
I assure you that the testing industry has a battalion of lobbyists—federal and state—to protect their business.
LikeLike
And pad politicians campaign coffers…
LikeLike
Standardized tests are a wasteful, expensive distraction. Stack rankings do not help young people in academics. The results of standardized tests are being misused by states to judge students, teachers and schools. As the study points out, formative assessments given by teachers are often useful tools that inform instruction while standardized tests highly predictable and rarely give us any useful information. The study also attributes our increasing poverty on our neo-liberal economic policies of the past forty years. Do we think politicians will pay attention and learn? Doubtful!
LikeLike
Thank you Diane. I’m working under a new, callow, not-real-bright assistant principal who holds what is essentially the diametrical opposite of your own developing and developed skepticism toward these instruments: she thinks these tests are the first and last word in education. It’s hard not to become demoralized in these circumstances, which is why I come back to this blog to hear the voices of reason resident here.
LikeLike
Thank you, Mark.
Give her a copy of my post. Or one of my recent books.
Anonymously.
LikeLike
“she thinks these tests are the first and last word in education.”
Obviously she doesn’t think.
LikeLike
Agreed. She is clueless. So sorry you have to put up with her.
LikeLike
Seems like every year we test the water and find out it’s wet. Now that we know that, we will switch to a new test and discover that circles are round.
LikeLike
The longer I taught, the more I questioned whether I ever knew what anyone actually knew. When I started teaching, I was full of hubris (and other substances we shall not discuss, a state of being which remains unchanged) and felt that testing held students responsible for their part of the bargain that the social contract demanded of them. Society held out the offer of education and the student was to interact with the teacher to become a contributing citizen within that social contract. Testing, my testing, held them responsible for their part of the deal.
The longer I taught, the less sure I was that I could actually assess whether a student was living up to the social contract. There grew in my thinking a real doubt about the reliability of information that would allow an objective view of whether I actually knew anything about what we had learned. Assessment is a paradigm of teaching that we are wed to, and I never completely supplanted my tests with other evaluation methods.
All this said, my involvement with standardized testing led me to build a real hostility to the idea. I once wrote questions for a State of Tennessee assessment test in Algebra. It was a joke at best, and some of my own questions that got on the test were very poor, a thing I knew even as the test makers for the state were fine with them. I also had a job for a short while editing and discussing test questions for a private company that had sub-contracted for the state. They were very unhappy with my contributions, which consisted of offering editorial suggestions about math test questions. They grew very tired of my insistence that the “standards” were not clear enough to justify a particular question. One of the people there became particularly exasperated with me.
Eventually the state prohibited teachers from knowing test questions. Laws were passed making knowledge of a test question an offence that could cause you to lose your teaching license. This occurred as the testing became more invasive in time and emphasis, and we reached a stage where it was obvious to any sane person that we had gone over the edge.
We are still over the edge.
LikeLiked by 1 person
I was on our school data team for about five years. We would pore through the data and central office admin would place some emphasis on test scores. The teachers would be very dismissive for the very reasons that were noted in the post.
Especially the “black box” and data release timing issues. I have no idea what they’re asking until years later. In high school social studies, it’s like a multiple choice trivia contest. (Plus the standards are so poorly written that it’s hard to tell what they often ask for.) Additionally, we test in April and get the results after those students have left us.
LikeLike
Here’s what I know and lived…Just the other day I was at one of our local restaurants where one of my former 8th grade student works (now a senior and heading to college). I told her if she needed any help navigating the “system” of college applications, I am always there for her. She said, “Mr. Charvet remember when Sabrina got the highest score on the STAR test? I nodded. “We were talking about it the other day and she said, ‘I just pushed buttons and didn’t even read the questions.'” As I have stated in many posts, I worked with kids who struggled in school. When testing time came around they already knew they would fail so just wouldn’t come go school. But, as I recall, the tests needed 95% of CBEDS testing in order for the tests to be valid. I had a rough going of just convincing them, “You know, these tests are not your lives. They never test YOU on what YOU know. And really, you have to realize you have to decide what is the MOST correct answer (and may not be the correct answer) out of some choices. AND, you never get to explain/justify your answers. As Diane pointed out, students who were in poor districts typically did poorly. Students in wealthy districts had the kids take the tests and moved on because, well, they already knew their scores would be high. I remember trying to get the kids in a very poor (kids living in shacks or trailers down old dirt roads) just to come to school and take the tests. One student told me, “Mr. Charvet, I don’t even understand the directions”. I told her to “guess” and be done with it. Then I said to myself, “Charvet, what the hell. So now you just want them to guess, so what good is that?” I also recall how guarded the test questions were. It felt like the scene from Animal House when the professor and his student went down in the basement, closed the shades, turned down the lights to smoke a joint. When I did finally saw what the kids were being asked to answer (Science especially), HOLY SNIKEYS! the questions were so far from what I had taught ( I mean I taught my guts out) but some of the terminology and subject matter was a “What the hell?” Geez. As typical my school scored a “1” and the school in Carmel, CA, well, a “10.” Gee whiz. But, if you asked my kids questions about agriculture, mechanics, or actually applied skills, they were “10s.” But that never happens. The kids who do well on tests are good at what they ask and it is typically reading and math. What about the art kids? I always thought, “Yeah, so you memorized facts. Can you actually apply those facts to something real?” I had a student who failed everything, but if you asked him to make something out of cardboard, he was an ace. In fact, I remember vividly him asking me if he could make a half pipe (skateboarding term) out of some old cardboard. The kids at that time played with TechDecks (miniature finger skateboards). His knowledge of geometry was phenomenal and there it was — a perfect scaled down version of a half pipe created with applied knowledge. But on his transcript, a math failure. When I was a kid, we had one big test at the end of the year. We took it to see what we had learned. And that was it. No one was nervous. No one stayed home from school. We all just took it and that was it. I don’t remember even seeing the results. At the end of the day, we all went and played at the park.
LikeLike
Rick Charvet,
I remember posting a few years ago about a professor in Texas who researched the tests and what they were testing and their predictive value.
He concluded that they were best at testing kids to get the right answers on the tests. I’ll have to find that post. I think he was punished for what he wrote.
LikeLike
@Diane— and many tests include questions that do not count but are used to “test” the test questions. I found from my own teacher credentialing tests, the “test” questions were exhausting.
LikeLike
These tests are always sold to the public as having an educational value. In Michigan these tests are given in late spring. Before the state mandated yearly tests we had our middle school students take the Iowa Tests of Basic Skills. They were quickly graded and we were able to use the sixth grade tests to determine which children needed extra support in the seventh and eighth grades. The eighth grade tests were used to help determine the individual student’s high school program and what courses of study would give the student the best chance of successful completion of high school and hopefully career opportunities for a successful life.
The state testing program ended our use of the Iowa Tests because the state’s tests required over a week to take, and even longer if a child was absent, thus requiring additional days for them to “make-up” the missed tests. The state takes so long to grade these tests that we would receive the results in the following November after our first trimester of the school year had ended. By that time we knew which children needed the extra help based on experience only. This makes these tests useless as far as helping teachers and counselors make the right educational choices for our students.
As teaching professionals we rapidly realized that, although educational researchers and professors of education saw a great deal of value in the data that these tests provide them, the real reason that the state legislators, mostly Republicans, wanted these test was to give them a reason to be able to call the teaching professionals in the public schools, especially in poorly funded rural and inner city schools, lazy and incompetent. This gave them an excuse to cut the state funding, or not increase it and let inflation do the cutting, for the public schools and to increase funding for the tax funded, privately owned, for profit, “Charter Schools.” Of course, since these Charter Schools were thought to be “highly innovative” and thus “excellent” schools which were not part of the state public schools system, these Charter Schools were and most still are exempt from the required annual tests of educational progress.
Of course for a public school to keep its state funding it must subject its students to a week or more of testing. The students know that these tests are meaningless and that affects their performance on these tests. The state legislatures tied not only district funding but teacher’s salaries, certificate renewal, and job security to these tests. This has resulted in teachers spending a large amount of time doing test preparation, which takes time away from the parts of the curriculum that are not tested or cannot be tested with an objective reference test. In the secondary social studies curriculum this meant that we no longer had time to teach the key skill of “critical thinking.” The fact that a generation of students were not taught critical thinking skills has made our electorate susceptible to demagogues like Citizen Trump.
LikeLike
Excellent comment, Ken Kolk. There is no question to me that the mandated testing (the horse race) serves the needs of the privatizers. They say, “Look at those terrible test scores. We have to save poor kids from failing schools! Parents know best! We don’t need certified teachers!” The floodgates open. Then come the attacks on teachers as groomers and pedophiles. Then the attacks on public schools as indoctrination centers. All part of the same hoax. Send public money to religious schools. Let everyone have a vouchers, poor or rich. A compete abdication of the state’s responsibility to provide a high-quality education to every child. Standards, accountability and professionalism are only for public schools.
LikeLiked by 1 person
The almost universal practices implemented around standardized tests is evidence that these corporate interests have a firm grip on education policy in every state. If Democrats turned these rascals out (I know, not likely) they might even route Republicans.
LikeLike
The big push for standardized testing wasn’t based on the desire to determine who well children were learning — the purpose was to use test scores as a tool for dismissing teachers and for selling the public on the “need” to privatize public schools.
The standardized-test-based “Value-Added Method” (VAM) of evaluating teachers has been thoroughly trashed by the very people who know the most about it: The American Statistical Association (ASA), the largest organization in the United States representing statisticians and related professionals, and they know a thing or two about data and measurement. The ASA slammed the deceptively-labeled ‘Value-Added Method’ (VAM) of evaluating teachers because VAM falsely claims to be able to take student standardized test scores and measure the ‘value’ a teacher adds to student learning through complicated formulas that can supposedly factor out all of the other influences and emerge with a valid assessment of how effective a particular teacher has been. But the ASA lays bare the fact that THESE FORMULAS CAN’T ACTUALLY DO THIS with reliability and validity. It’s pure political ideology to claim that VAM based on student test scores reflects teacher effectiveness.
In its official statement, the ASA points out the following and many other failings of testing-based VAM:
“System-level conditions” include everything from overcrowded and underfunded classrooms to district-and site-level management of the schools and to student poverty.
A copy of the VAM-slamming ASA Statement should be posted on the union bulletin board at every school site throughout our nation and should be explained to every teacher by their union at individual site faculty meetings so that teachers are aware of what it says about how invalid it is to use standardized test results to evaluate teachers or principals — and teachers’ and principals’ unions should fight all evaluations based on student test scores with the ASA statement as a good foundation for their fight.
LikeLike
It shocks me that even among some commenters on this blog, there are those who seem to think that the state standardized tests somehow measure what they purport to measure. In ELA, this definitely is NOT the case. The tests do not validly measure what they purport to measure, and that is DEMONSTRABLE. Here:
The breathtaking irony is that the test proponents, who claim to be all about the data and scientific rigor, are utterly unaware of how invalid the ELA tests are even though only a tiny bit of rigorous thought about this matter would expose the tests for the pseudoscientific quackery that they are.
The Math tests are slightly better but have their own issues. I’m not going to go into those here, as they are complex.
But please, please, please read this essay about the validity of the ELA tests.
Also note that almost always journalists write about how our schools are doing, they refer to these state tests. They naively take it for granted that the tests measure what they purport to measure. They do not. Read this essay to learn why, if you haven’t figured this out on your own already. Many of you here have doubtless figured this out by now. People should stop writing as though the tests were valid but their use is the problem. The tests are pseudoscience. They are an extremely lucrative SCAM.
They should be ended NOW, and then a truth and reconciliation committee should be established to recover from the testing companies the funds they have gotten from the scam.
LikeLike
cx: Also note that almost always, when journalists write about how our schools are doing, they refer to these invalid state tests as evidence. Wrong from the start.
LikeLike
Read the study linked to see why math tests are just as worthless as English tests.
LikeLike
I will be interested in this. I have my own expose to write about the federally mandated state Math tests. I will eventually get around to this, but I haven’t yet because its a complex topic that will require a lot of explanation. So, I will look at that article with interest.
LikeLike
cx: it’s
LikeLike
I think that you are talking about this study:
https://www.mdpi.com/2227-7102/14/2/129
Maroun and Tienken, on the one hand, and I on the other, are addressing two DIFFERENT topics., Their study, like the ones done by Richard Rothstein of The Economic Policy Institute, addresses the question of the correlation of test scores to economic status and ZIP Code as a stand-in for economic status. Mine is an ENITRELY DIFFERENT TOPIC. My essay is about the VALIDITY OF THE TESTS THEMSELVES. I address why they don’t measure what their makers say that they measure–proficiency with regard to the standards.
Different topic.
LikeLike
Yes, that one: https://www.mdpi.com/2227-7102/14/2/129
LikeLike
The state “standardized”* ELA tests do not measure what they purport to measure, which is proficiency with regard to the ELA standards. Here’s why: each CCSS or state standard, typically, is extremely broad. On a given test, there will be at most 1 or 2 multiple-choice questions about some of the state standards. Many of the state standards will not be tested at all because there are more standards than there are test questions, typically. Now, consider this standard:
The student will make correct inferences from written texts.
So, here’s your assignment, dear reader of this blog: write ONE multiple-choice question that will determine with certainty that the student has or has not gained proficiency in this standard, which covers all kinds of inferences for all kinds of written texts. Try again with this standard:
The student will use in his or her writing proper grammar, usage, mechanics, and spelling.
Again, write ONE multiple-choice question that will validly determine whether the student has achieved proficiency in this standard.
Not possible, right? It is LITERALLY IMPOSSIBLE to write one or two questions that validly test standards that are so broad. I mean actually impossible, like attempting to draw a square circle or to build a perpetual motion machine.
That this is not obvious to anyone who has thought for more than a moment about these tests is beyond my poor imagination. But it isn’t, it seems, to a LOT of people who consider themselves devoted to data and rigorous thinking.
That would be laughable if it weren’t so tragic. It’s a kind of blend of tragedy and farce. And the joke is on taxpayers, students, parents, teachers, and administrators as the test manufacturers laugh all the way to bank, after stopping by to pay off some state and federal education policy makers.
LikeLike
*On those scare quotes around “standardized”: The state test makers do not put scores through the procedure for converting them to standard scores, or Z-scores, so these are not technically standardized tests, despite the ubiquitous improper usage of “standardized” to refer to them. The state tests pretend to be criterion-referenced tests (tests of absolute achievement). The criteria referenced are the standards. So, they are a different kind of test altogether than are norm-referenced standardized tests like, say, the Iowa Test of Basic Skills.
LikeLike
Z-scores, aka standard scores, reframe raw scores as number of standard deviations above or below the mean. State test scores are not so standardized, so these are not, technically, standardized tests.
LikeLike
This conversion to Z-scores is done because the purpose of true standardized tests is to compare performance among students and group of students. Criterion-referenced tests like the state exams in ELA and Math have as their purpose not comparison but determination of whether the student or group of students has met some criteria–in this case, shown proficiency with regard to particular state standards. But as I just demonstrated, above, the state ELA tests DO NOT DO THIS. THEY CANNOT DO THIS given how broad the standards are and how few questions are asked per standard. These tests do not do what they purport to do. They are scams. Claiming that they are valid measures of the standards is simply false. They aren’t.
LikeLike
I’m a liberal, retired HS teacher, and I agree with & accept what Diane & other experts say about the economic disparity affecting students differently, whether related to testing or to academic classes in general. However, I have always questioned how & if students should be tested & evaluated differently from different areas of the country, as well as different economic backgrounds. I believe that there has to be some type of standard which all of our students must achieve in order to be able to get into colleges, tech schools, and eventually jobs, which can then assure those institutions that those students are qualified. It gets very sticky, not to mention playing into the hands of righties, when liberals look to set different standards for different levels of the population. It goes to the argument of–do we want the best, most qualified person for the job, school, organization, or do we need to let in a variety people in to those entities, not just the top candidates? It’s a tough question that all of us need to grapple with and be brutally honest with, as well.
LikeLike
The state ELA tests are not valid measures of achievement. See my explanation of why that is so, below.
LikeLike
above, rather
LikeLike
I agree. I was a public HS teacher in South Florida from 1987-2020. I guess I’m just flashing on my early days coming up & being told that the SAT & ACT tests would be able to equally compare students from big city schools to tiny rural schools. Local schools and their grading systems may be easier, more lenient or lacking in standards compared to other schools, so a nationalized test could equally judge all students throughout the country. Sure, current state tests are crap & poorly structured, but don’t we ideally want a some type standardized test to be able to judge & evaluate all students against certain standards?
LikeLike
The SAT and ACT are standardized tests. They are meant to be used as comparative instruments. The state tests are criterion-referenced, nonstandardized tests.
LikeLike
loved2teach– This reminds me of the argument over whether SAT/ACT tests should be optional, or perhaps not even used, in college admissions. The studies have long shown that SAT/ACT tests are predictive, max, of freshman year grades; a couple say 1st semester-only grades. Nothing else, not even likelihood of graduating. Whereas high school GPA is a fairly reliable predictor of success in college and likelihood of graduating.
The naysayers maintain that GPA from one high school is not equivalent to GPA from another. But that’s irrelevant in an important sense: admissions staff examine GPA’s in context– how challenging are the courses, what are grades in courses closely related to the field students want to major in, and of course, what high school was attended. Admissions staff have to know that territory or they couldn’t do their job.
And of course the whole argument is irrelevant in the sense that 50% of our colleges aren’t selective at all, and another 25% are not very selective. Only 10% of our colleges are so selective that they require every possible measuring stick they can get: SAT/ACT, GPA, high quality essay, demanding extracurriculars, awards, etc. And like it or not, family wealth, for those highly-selective colleges that do not have the ability to pay all costs for those who cannot.
Whether there could be such an animal as a test that can separate wheat from chaff for the nation’s entire sr high school class is debatable. One problem we have: every state has its own ed standards. At this point the SAT [ACT too?] is CCSS-based. Sure, a lot of states have tweaked/ rebranded since 2015 ESSA law, but still have stds that reflect CCSS. But some states have completely revamped. Plus the secondary problem: as Bob points out, at least in verbal section, the CCSS-like standards cannot be tested adequately at all. And, if one extrapolates from the study linked to this post, the math section is too married to reading comprehension/ general knowledge to be reliable.
Considering all we already know about the reliability of standardized tests, do we want to aim for stds/ stdzd tests that you suggest, hypothesizing the possibility of a reliable measure for “college/ tech school/ career-ready? I’m not so sure. We already have 62% of US high school grads going on to 4yr college. Our economy seems to indicate/ demand a higher %age. That would require getting an even larger proportion of lower-middle/ working class college into tertiary ed than already do– the very people least likely to do well on stdzd tests. My sense is that the path ahead indicates (1)a need for more tech/ trade training in both secondary & tertiary ed, and (2)maximize tax support of tertiary ed, & find ways to lower its costs.
[Someday I hope to do a deep dive into French bachau & other Euro college admissions tests to understand what they are trying to accomplish, and how well it works.]
LikeLiked by 1 person
Bravo
LikeLike
p.s.– Another problem I have with your hypothesis– again, knowing what we already know about standardized testing– the theory behind it automatically would reinforce inequality between high income & low income people. And another: today’s (& yesterday’s) standardized tests for high school completion/ college admissions are purely academically based. They cannot hope to, & don’t even try to assess technical skills [tech school-/ career-ready], not do they assess musical/ artistic skills, which are not necessarily accompanied by high academic skills.
LikeLike
Thanks, Ginny!
One of the horrors of No Child Left Behind and the Common Core testing regime was that it created one category for instruction of all students and labelled it education for “College and Career Readiness.” This was PROFOUNDLY IDIOTIC. And to kick off this idiocy, Bushie Jr., aka Shrub, dramatically cut funding for vocational education. Every kid was supposed to go through an identical milling machine, as though kids were all the same and as if an incredibly complex, varied, pluralistic economy didn’t need to have people’s DIFFERING talents identified and nurtured in school. Combined with the purchase of the GED by Pearson and their making it FAR MORE DIFFICULT in order to force kids to take it numerous times in order, perhaps, to pass it so that they could then go on to a tech or vocational or apprenticeship program, this was A DISASTER for millions of kids.
Screws, nuts, bolts, nails–these need to be standardized. Kids? Are YOU FREAKING KIDDING ME???!!!!
Oh the evil that was done there, and it is almost never mentioned, so thank you, thank you, Ginny, for bringing it up.
LikeLike
Read! Don’t be fooled by marketing. Read!
I say to you, read! I say to my self-enraptured principal, read! I say to my side dealing superintendent, read! Not advertising, peer reviewed research.
*”Jamil Maroun and Christopher Tienken have written a paper that will help you understand why standardized tested is fatally flawed. The paper is on the web and its title is:
The Pernicious Predictability of State-Mandated Tests of Academic Achievement in the United States.”*
A few times a week, my students, their families, and I receive email from my school district selling online test prep products falsely purported to boost scores. Dupes give up their data using the products. Read!
LikeLike
Note also that Diane, like Rothstein, Maroun, and Tienken, has written extensively, in several books, about how test scores predict ZIP Code–are really great measures of ZIP Code. The point to so-called “Reformers”: it’s about the poverty, stupid. My subject is not that. It’s the tests themselves and why they are so unreliable.
Here’s why we are stuck in the testing miasma: Actually knowing why the tests do not work requires learning that politicians, journalists, and unfortunately, a lot of education bureaucrats and administrators are not willing to do. I encourage people to read this piece so that they understand why the state ELA do not work and then to share it widely. People need to know this. The tests do not do what their makers say they do.
I am totally flummoxed as to why people don’t understand that the lack of the validity of the tests is extremely important. They are a scam. Getting people actually to read my explanation of WHY THEY DON”T WORK is almost impossible, weirdly. It’s like no one wants to bother to make the effort.
But like a magic elixir for curing cancer sold by a Wild West traveling medicine man, these tests do not do what they purport to do. They are invalid. They are a scam. And the results of them are not, for that reason, “data.” ROFL.
LikeLike
My subject is not that. It’s the tests themselves and why they are so invalid.
LikeLike
Dr Strangelove: How I Learned to Love the Tests.
LikeLike
I am at the point at which I totally despair of ever being able to make any headway against the state standardized tests, the Common Core State Standards (CCSS), and the many sets of current state standards based on the CCSS. I have been at great pains to explain in detail why the state ELA tests are not valid, why they do not test what they purport to test. I have written ever since the CCSS first appeared to explain why they are garbage, as the state standards that preceded them were garbage. My overwhelming sense is that almost no one is bothering to take the time to consider my arguments, my detailed, careful arguments. Journalists and politicians and educators and even people on this blog continue to treat the scores from the state tests as though they were valid measurements of student achievement. Sure, a few complain about various matters related to the tests, that they cost a lot, that they lead to teaching to the test, that the information from them comes too late to be actionable, etc. etc. etc. blah blah blah. But no one seems to be willing to take even the most minimal amount of time to follow the arguments that explain why these tests do not do what they purport to do, why they are pseudoscience, why they are an elaborate, costly scam. Almost no one, even among those who oppose the tests, even among those who oppose the standards. Why? Too much work? I don’t know. I am so tired of beating my head against this wall.
LikeLike
The federally mandated ELA tests are invalid. This means that. . . .
–they do not actually measure what they purport to measure.
–they do not provide information about individual students, schools, districts, states, or nations that rises to the level of “data.”
–scores from them cannot accurately and responsibly be used as a measure of how students are doing in reading and writing in legislation, regulations, news stories by journalists, white papers or essays by pundits, or studies by researchers and scholars.
–scores from them cannot accurately and responsibly be used by teachers, administrators, and district officials to gauge student performance and design curricula, including remedial curricula.
–scores from them cannot be used accurately and responsibly by curriculum developers, including educational publishers of online or print materials, to prepare new curricula
–scores from imitations of these tests built into criterion-referenced online educational tech cannot accurately and responsibly be used to gauge student progress and place them at a proper level in the programs’ instructional sequences.
–scores from them cannot accurately and responsibly be used by commenters as gauges of how students in particular demographic groups or places are doing or how they are doing in relation to other students.
Because the tests are not valid, ANY TIME ANYONE USES THE SCORES FROM THEM IN AN ARGUMENT, he or she is acting irresponsibly and without scientific warrant based on inaccurate, invalid information.
THIS IS WHY WHAT MATTERS MOST IS THAT THE TESTS ARE NOT VALID, that THEY DO NOT MEASURE WHAT THEY PURPORT TO MEASURE.
LikeLike
It is tragic that self-style education “reformers” like Bill Clinton, Jeb Bush, Rod Paige, Bill Gates, David Coleman, Arne Duncan, Michelle Rhee, Cambell Brown, Michael Pirelli, and the like RUINED the use of perfectly decent terms like data, rigor, and accountability by using them to refer to invalid tests, test scores, systems involving invalid tests, and evaluation systems based upon those that do not involve actual data or rigorous assessment instruments or analyses that are not themselves accountable for failing to provide accurate information or information that was actually useable to improve learning outcomes.
Let me expand a tiny bit on that: decades, now, of the standards and testing “reforms,” including vast changes in standards, curricula, and pedagogy nationwide, has led to ZERO improvement in leaning outcomes by the “reformers” own measure, scores on the federally mandated ELA and Math tests.
LikeLike
Dear Diane Ravitch,
It is time to let everyone know that, beyond high-stakes norm-referenced testing, there exists a truly educational assessment and evaluation that is both humane and productive.
Check out: NGSSrubrics
I look forward to sharing this paradigm with you,.
_________________
Paul Zachos, PhD
Director, Research and Evaluation
Association for the Cooperative Advancement of Science and Education (ACASE)
110 Spring Street Saratoga Springs, NY 12866 |
paz@acase.org | http://www.acase.org
LikeLike
Dear Paul,
Please share the assessment tool that is both humane and effective.
LikeLike