Back in 2009, when Arne Duncan announced the Race to the Top competition, he said we as a nation would literally be “racing to the top” of international competition by adopting his favored ideas: expanding charter schools, evaluating teachers to a significant degree by the test scores of their students, “turning around” low-scoring schools by radical measures such as closing them, creating state and national data storehouses to track students, and adopting “college and career-ready standards” (aka, the Common Core). Almost every state fell in line, because they had to do what Arne wanted in order to be eligible for a share of $4.35 billion.
But the report cards have not been kind to these “reforms.” When the National Assessment of Education Progress issued its regular report in 2015, test scores were flat or declining in most states.
Now the latest international test scores are out, and the U.S. has made no gains. We are not racing to the top. We are standing still. Why? Because Race to the Top did not address the root causes of academic failure: poverty and racial segregation. Charter schools have produced marginal gains at best, with some far worse than public schools. Evaluating teachers by test scores has been an abject failure, criticized by the nation’s leading scholarly organizations, including the American Statistical Association, which is not an arm of reformer-dreaded teachers’ unions or the “status quo.”
Here is today’s report from politico.com:
PISA RESULTS: BAD NEWS IN MATH: American 15-year-olds are getting worse at applying their math skills in the real world, when compared to their international peers. The 2015 Program for International Student Assessment results are out and they show a drop in “mathematics literacy” scores for U.S. students since 2012 and 2009. “Of particular concern is that we also have a higher percentage of students who score in the lowest performance levels … and a lower percentage of top math performers” compared to the international average, said Peggy Carr, acting commissioner of the National Center for Education Statistics, which released the results. The disappointing numbers come after results on another international study – the Trends in International Mathematics and Science Study – recently showed gains made by U.S. fourth and eighth graders in math since 1995.
– U.S. science and reading literacy scores weren’t much different from previous years. Boys outperformed girls in science and math, while girls outperformed boys in reading. Scores for Massachusetts, North Carolina and Puerto Rico were broken out for international benchmarking purposes, and revealed that Massachusetts students, on average, are outperforming students in the U.S. and worldwide in all three subjects. North Carolina students were comparable with U.S. average scores and Puerto Rican students fared worse. PISA measures the performance of 15-year-olds every three years in three subjects across dozens of education systems worldwide. Check out the results here .
– Education Secretary John B. King Jr. is in Massachusetts today to hail the state’s success with PISA – while noting that the nation as a whole is “losing ground.” According to prepared remarks, King will say that it’s “a troubling prospect when, in today’s knowledge-based economy, the best jobs can go anywhere in the world. Students in Massachusetts, Maryland, and Minnesota aren’t just vying for great jobs along with their neighbors or across state lines, they must be competitive with peers in Finland, Germany, and Japan.” King will say that Massachusetts embodies the importance of perseverance. “The PISA results announced today for Massachusetts didn’t happen instantly or by accident,” he’ll say. “It has taken years of people showing courage – principals, teachers, parents, students, and state and district leaders. It has taken years of overcoming challenges. It has taken years to make real and meaningful change happen. And it will take time to see the work we are continuing to do today truly pay off for students.” More on King’s visit.
– Other noteworthy highlights: U.S. students value a career in science and have high expectations of having a science career, but they’re falling short when it comes to skills. Countries like Finland, Germany, Switzerland and Japan are also seeing better student outcomes than the U.S., while investing fewer hours in actual teaching – giving teachers more time for professional development and advancing their careers.
As I have often written before, the international test scores do not predict the future of our economy or anything else. Scores on standardized tests measure family income and income inequality. If you want to know more, read my chapter in “Reign of Error” on international tests and what they mean and do not mea.
Diane Ravitch's blog 
But, Diane, that’s so unfair! How can you blame Secretary Duncan when he’s not even the head of the Education Department any more? Anything positive is because of his changes. Anything negative is because of bad teachers and failing schools.
Great satire, Ohio Algebra II Teacher.
Ohio teacher,
Arne got rid of all those bad teachers and failing schools during his seven years as Secretary with $5 billion in discretionary funds and waivers that he withheld if states didn’t comply.
And they thought marketing a BAD IDEA and “motivating learning” as if it is a race would work. Duh…how ridiculous. Take every law regarding education since Reagan and turn it around; Therein lies the truth. The laws and Acts have just gotten more and more ridiculous.
Wish the oligarchy would “ping on” college and professional sports! Lots of damage is done in this arena. But no, the oligarchy “owns” sports teams. $$$$$$$$$$$!!!!!!! Reminds me of the Roman Colosseum where only mayhem and death prevailed.
You say the ‘reforms’ failed because they failed to take into account the root causes of poverty and racial segregation, I’d like to add they also failed to take into account real teaching and real learning. I know teachers in poverty schools who have had to resort to what they call “sneak teaching” to help kids understand complex concepts. Great teachers sneaking powerful learning opportunities for their children because administrators are observing for curriculum alignment and teaching to a test – so absurd.
You are so right!
Nicely put!! Here’s to sneaky teachers!!
“Great teachers sneaking powerful learning opportunities for their children. . . ”
“Great teachers” (whatever that supposedly is) do not “sneak powerful learning opportunities” in their classes. Were there such a thing as a “great teacher” (surely an edudeformer meme), he/she would be providing “powerful learning opportunities” for all students out in the open all day and refusing to teach using educational malpractices that those adminimals mandate.
Don’t expect me to challenge anybody on Pedagogy. I gave up on becoming a teacher after my first ed course . I am in full agreement that poverty is the determinant. The quickest way to eliminate it is Jobs. “Little Hands” doesn’t get everything wrong. Curiously I wonder how our Deindustrialized Mid West is doing as compared to other regions if the OECD even breaks that out. How they are doing compared to 10-16 years ago.
Looks like I might have to read Newsday on long island to hear the editorial staff whine once again about failing schools and failing teachers. Instead of the failing oligarchs that own the paper. Time for another letter to the editor.
Of course what they will refuse to admit is when Roslyn school district was allowed to participate in an experimental PISA program it out scored or tied Shanghai . What a shock. I wonder why ?
The Chicago Tribune wrote a fawning editorial about DeVos. I didn’t realize how conservative the Tribune is.
I get the Trib. They do some good reporting. Their editorials are frequently impossible to read. They are a mouthpiece for the most conservative voices in the state. Our governor, Bruce Rauner, praised DeVos as well. Not surprising. I don’t know if we will survive him. I’m afraid the most vulnerable won’t.
But the administration hasn’t stopped praising its strange bedfellow allies, no matter how abject their RttT related failures. Bill and Melinda Gates were just given the nation’s highest civilian honors, Presidential Medals of Freedom. So were some pop singers because the Pres. likes their singing, some Hollywood actors because he likes their movies, and some NBA basketball players because he likes their dunks. What a joke! No matter how bad the next four years may be, it doesn’t erase the pain we’ve suffered during the past eight. They in power don’t get it, or they don’t care.
Massachusetts? Isn’t that the state with the highest percentage of those greedy, sex crazed union protected teachers? How can that be? I thought unionized teachers were the reason why US education is so bad. Oops.
“According to prepared remarks, King will say that it’s “a troubling prospect when, in today’s knowledge-based economy, the best jobs can go anywhere in the world.”
It is a more troubling prospect that in this knowledge-based economy that as the leader of America’s public school system. John King promoted a set of national standards and compulsory testing that completely ignored content knowledge in favor of empty skill sets. Even more troubling is that King had seemingly no idea that the Common Core standards he was pushing here in NY and then on a nationwide basis, were just so vapid.
I don’t think they are vapid, but the timeline in which they unfold is NOT developmentally appropriate. And when you tie them to high stakes testing, then the skills become more important than the content instead of having parity with the content. This is subject to intense review and reform, I think.
Norming always helps. It’s critical.
The Common Core standards actually have a good part to them: the pages that say that schools need cumulative, coherent, content-building curricula. The problem is that virtually every district in the nation is completely ignoring this part of the standards and fixating instead on the ill-conceived skills portions of the document (which imply that the ability to read a complex text is the result of some complex text reading skill that can be built up with practice; that’s not how reading works. All the skills practice in the world will not enable you to understand “The Wealth of Nations” if you don’t have a big vocabulary and a basic grasp of how the world works).
Today’s New York Times has a piece by Amanda Ripley about the latest PISA scores. She quotes Andreas Schliechter, PISA’s head, as saying he thinks Common Core will eventually boost US scores. If we implemented the content part of the standards, he would be right. But he probably does not realize that this is being universally ignored as an irrelevant and incomprehensible footnote to the standards.
Ripley shows herself to be ignorant of the latest cognitive science when she writes:
“Every three years, half a million 15-year-olds in 69 countries take a two-hour test designed to gauge their ability to think. Unlike other exams, the PISA, as it is known, does not assess what teenagers have memorized. Instead, it asks them to solve problems they haven’t seen before, to identify patterns that are not obvious and to make compelling written arguments. It tests the skills, in other words, that machines have not yet mastered.”
The ability to think is based heavily on memorized information, as Hirsch shows by citing studies in cognitive science. Problem-solving ability in a given domain depends on prior knowledge in that domain. Studies show that top chess players deploy memories of prior chessboard configurations when making decisions –not raw reasoning/thinking power, as intuition and conventional wisdom would suggest. So the kids who score well on PISA problem solving tasks are likely the ones with the most robust, general knowledge base, not those with hypertrophic thinking “muscles”. PISA is thus a knowledge test in disguise.
Ponderosa, I agree. It’s hard to execute skills without background knowledge. And as a history teacher at both the high school and community college level, I’ve found that students are truly engaged by the facts of history more so than the skills around it.
I have had students tell me that they often feel like they’re being to complete tasks without enough content to properly complete the tasks.
Our PISA scores would rise, and the achievement gap would shrink, if we shifted to a coherent content-focused curriculum, instead of the skills-based curriculum we’ve hewed to for the past 4-5 decades. Proof? France had a coherent, content-focused curriculum until 1989 when it shifted to an American-style skills-focused curriculum. Scores went from top-of-the-pack to mediocre. Sweden did something similar in the early 90’s and scores sank. Germany did the reverse and scores rose. Japan, China and Korea have never deviated from the coherent, content-rich curriculum and scores have stayed consistently high. When will we wake up and realize that if you actually teach kids stuff they’ll get smarter? Instead, we carry on with this endless failed experiment in progressive, anti-knowledge, skills-focused, sage-off-the stage pseudo-education. Want to learn more? Read E.D. Hirsch’s latest book, “Why Knowledge Matters”. All teachers need to read this book to undo their baleful ed school brainwashing.
By the way, a knowledge-rich curriculum is a proven way to shrink the achievement gap between demographic groups (see France, Sweden and Japan). Progressive, skills-focused curriculum is a proven way to widen the gap. Does evidence matter to us?
There is a balance between content and skills that we have lost. We have gotten to impressed with the ease of finding information in the information age. My education was too far the other direction. I was told what to learn and expected to spit it back. That’s a bit of an exaggeration, but it was college and graduate school where I really discovered how to think for myself. Give me the content, but then teach me to think about it, too.
2old,
Blooms Taxonomy: the fat part on the bottom of the pyramid is learning facts. The skinny part at the top is higher-order thinking. Translation: early in life we need to load up on facts. Later in life we can do fancy thinking with that mass of knowledge. But, ludicrously, many educators read the taxonomy this way: facts bad; higher-order thinking good.
I have read Bloom’s taxonomy. It makes basic sense. However, thinking about learning and how we learn does not and should not wait until the later years, nor does Bloom imply that it does. We learn a little, we experiment a little; we learn a little bit more; we experiment some more. Kids learn about the metamorphosis of a caterpillar into a butterfly and the various stages involved. They watch caterpillars change. Someone tries to help a butterfly emerge from the chrysalis and discovers that by “helping” they have crippled it. We need to interact with what we are learning. How we (are ready to) interact will change and mature as we do, and, yes, our teachers, coaches, and parents may actually have suggestions on how to practice or utilize what we learn. Practicing a skill, however, without content on which to practice makes as little sense as learning content without learning/knowing how to use it. It is a balancing act between knowing, feeling, and doing. I have an idea that we basically agree; we are just coming at it from slightly different angles. Perhaps because I was a special education teacher, my focus has been on how I can help the kids learn how to learn: how to access the content. Those tools without the content make no sense just as content with no tools to use it is equally nonsensical.
Of course just like VAM, there is not necessarily a direct correlation. Even if education were doing all the right things and improving, who’s to say that distractions by entertainment and technology, smart phones, and devices aren’t taking up more time out of students lives? Smart phones are so recent, I don’t think the negatives on students focus or concentration on learning are fully known.
RTTT is rearranging chairs on the ship. Entertainment, trivial knowledge, and photo image addiction is the iceberg.
I’m not so pessimistic. I can still engage 12 year olds with graphics-heavy slide lectures about simplified, clear explanations of history. Direct instruction, done well, can still transmit important knowledge to young minds despite the lure of social media and the rest. (Now getting them to resist the temptations at home and do their HW is another matter). The bigger problem, to my mind, is that teachers are told not to even try to transmit knowledge. So wrong-headed!
TC, everything you said, plus the stunted attention spans (something I see in myself, as well) caused by the devices our students are captives of.
As things proceed we will find out, too late, that the Overclass will have taken over the analogue, physical world, and left the digital simulacra for the proles.
Actual unpolluted beaches, mountains and rivers, ust like real, healthy food, will only be accessible to the rich, and the rest of us will be left with high fructose corn syrup, soylent green and digital representations of what planet earth once looked like.
@Michael Fiorillo. Devices are not what causes the stunted attention span…it may be a factor. Bodies are made to move especially when awake and when our bodies don’t move, the mind takes over. If you want a kid to sit still and pay attention, the best way to accomplish that is if the body is worked first. Loss of recess (or 15-20 min recess) and less PE causes the stunted attention span. If their bodies aren’t moving, then their minds take over and their minds need constant stimulation. This even happens with adults.
As a science teacher of 25 years I don’t believe the “root causes” are poverty and segregation, but breakdown of the nuclear family, decay of morality and integrity and an overall hedonistic-consumerism-materialism/atheism.
I work with poor Haitian kids, poor Latin students here in Miami, and many of them excel, because they have not been through 1-3 divorces, having loving and supportive parents who nurture, care and discipline them.
I have rich White kids who have the opposite, and they do poorly in my classes.
So, class, color or integration may have some roles, but they are not “root causes”.
Time and again research shows that the number one predictor of high school success is the family support structure. Yes, poverty effects families, and race can be segregated along socio-economic divisions, BUT a strong and supportive family can overcome all of these, and the absence of one (or degradation of one) cannot be made up by RTT monies, school choice or any other physical tool we try to use to solve an essentially spiritual, moral, problem (though most humanistic sociologists will focus on external factors, not heart/conscience factors).
Rick, I don’t have a lot of time. I agree with much of your post. I was asked this summer, by a non-teacher friend, the following question: “If you could identify one problem that you believe is most responsible for America’s decline, what would it be?”
My answer: The breakdown of the nuclear family. I’m not blaming anyone or anything about it because there are many reasons that one could list in sociologically and economically. But it’s a huge problem.
Rick: I taught poor Latinos and Haitians in New York for many years. We had lots of success with these students despite poverty. Their families were stable and an anchor for them. These students were aspirational unlike some of our American minorities that suffered from a fractured family and substance abuse in the home. A strong family is a security blanket.
I agree very much with your experiences and assessment. At the best of times the caring teacher is the best lifeline for the disoriented student and a light for them. How then to provide the best environment for more teachers to be more of these kinds of teachers, and students to stay fixed on them? This to me, is one of the great questions that everyone is trying to answer and work out.
“. . . and an overall hedonistic-consumerism-materialism/atheism.”
As I am wont to say: Horse Manure!
Your confusing and confounding atheism with “hedonistic-consumerism-materialism shows a fundamental lack of knowledge of what atheism is and also shows an arrogance and smugness of those believers who believe in a god of ancient Middle Eastern tribal myths of their own supposed righteousness over such supposed heathens.
Gimme a heathen over a righteous mythological believer every day!
Do you teach creationism, Rick, in your science classes?
Duane, I didn’t mention the name or doctrine of the theism you seem to detest. All I pointed out is the obvious and historically verified correlation between a people’s belief system and their practice. Look at what happened to Russia and China, or Cuba, when traditional theism (Judeo-Christianity, in this case) was kicked out and banned by the atheistic, evolutionary materialism, paradigm of the communist party. Look at Naziism’s rise in Germany, fueled by a rejection of Judeo-Christianity and a belief in sociobiology and the applied fallacy of “competition for the fittest” by HItler and his ilk.
You can huff and puff all you want, name call me, try to put me in a box so you can discredited obvious inferences….but the facts speak for themselves.
Now that I think of it, I suppose it must be difficult to trust any climate change model that doesn’t properly factor in extraordinary climate changes that caused the Great Flood that is chronicled in Genesis.
Not righteous here, Duane. I’m intensely aware of my errors and sins, and that is why I have, and need, Faith. “I didn’t come to call the “righteous”, but sinners to repentance”. Give me a humble person who realizes they are not god/perfection, over the arrogance of a Neitzchian “super-human” materialist any day. Do you teach Naziism (a logical consequence of evolutionary theory applied to sociology) in your classes Duane?
Considering that I taught HS Spanish, I couldn’t work Naziism into the curriculum. Where were you to help me out with that? I guess I might have tried to advocate for Francisco Franco as an example of Naziesque dictatorship but he was not contemporary enough for my students to have heard of. As Chevy Chase was known to have reported:
Sorry to push this thread further off-topic, but since you and Duane started off down this road already, I’m curious to ask what your view is on climate change. I’m certainly no climate scientist (or any kind of scientist), but it would seem to me that if the earth were only 10,000 years old, that would call into question much if not all historical modeling used in the field of climate change science. Certainly, if the earth were 10,000 years old, ice ages could not have been occurring hundreds of millions or billions of years ago. Does your belief that there is good evidence that the earth is only 10,000 years old (or thereabouts) have any impact on your confidence in climate change models and our ability to understand possible causes of climate change and make predictions about future climate change?
“Do you teach creationism, Rick, in your science classes?”
If I recall correctly from a comment thread a while back, he teaches that there are facts that are incompatible with evolution (not to mention geology and physics/astrophysics), and then teaches them that they should make up their own mind about whether humans evolved and whether the earth is billions of years old or something closer to 10,000 years old. If I had to guess, I’d guess there’s also some talk about how the existence of a watch implies the existence of a watchmaker, etc.
“Time and again research shows that the number one predictor of high school success is the family support structure.”
Do you have some links for us to those studies, Rick?
There is a plethora of them, search for yourself. Seems self-evident.
Don’t have any, eh?
No, but apparently you don’t have the desire to find them on your own. In my doctoral program we studied several of them. So, you deny to primary role of family input and support? You believe “the village” can make up for the lack of family (when all sociological studies support the opposite). Yes, people can overcome their lack of family, or rise higher than family degradation would predict. Yes, with personal will and fortitude one can overcome many limitations, but that doesn’t mean we deny the factor limitations play. So, take your antagonism and channel it into a literature search, and answer your own questions.
You are putting words in my mouth, Rick. I never said anything about denying the primary role of the family or that “the village” can make up for the lack of it. Your words not mine.
I just asked for some sources to back up your statement and you haven’t provided. So your opinion is just that an opinion that you are pawning off as fact.
My “antagonism” (again, your word) comes from what I perceive as a not so subtle effort by you to denigrate those who do not share your opinions about morality, a morality that is mediated through a non-existent mythical god. And I don’t buy that take on ethics and morality.
Again,,,,you seem too lazy and go do the literature search. For every assertion you make, do you provide literature citations, and if not, then don’t be a hypocrite.
And… if you cannot provide a better “myth” for me to believe in (because to the skeptical constructivist all narratives are myths, there is no absolute truth), then let me have mine and your can have yours (oh, but your’s is not a myth, so somehow you’ve found absolute truth, and you get upset when others speak of theirs???). If we all just have our myths, then why get offended or worked up about someone else’s, if there really is no external truth or being out there?
I’m rather amazed. There’s all this posturing – from commenters here and from the mainstream press – that PISA and TIMSS scores of 4th and 8th grade students really mean all that much. Guess what? They don’t.
PISA scores (the scores usually cited by public education critics) are quite sensitive to income level. If one disaggregates U.S. scores the problem becomes clearer: the more poverty a school has, the lower its scores.
The late Gerald Bracey, wrote this about PISA testing:
“Even if the tests were valuable sources of information, there would be the fact that among 21 developed nations, the U.S. had the highest poverty rate. If we’re number one in poverty, is it reasonable to expect that we would be number one in test scores? But, really, the tests aren’t informative. In fact, if you analyze the test scores by the poverty levels of schools, the top 30 percent of American kids score higher than the highest country in reading. And another 28 percent score high enough that if they constituted a nation, they’d rank fourth in the world (out of 35).”
And this, also on PISA:
“Worst of all, PISA uses a statistical technique called the ‘One Dimensional Item Response Theory.’ Joachim Wuttke of Jülich Research Center in Munich contends that this is wholly inappropriate. ‘Items that did not fit into the idea that competence can be measured in a culturally neutral way on a one-dimensional scale were simply eliminated.’ This was 65 percent of the items in the field tests. This corroborates the University of Oslo’s Rolf Olsen who argues that ‘in PISA-like studies, the major portion of information is thrown away’.”
Singapore and Hong Kong topped the PISA and TIMSS charts. Andreas Schleicher, head dude for PISA, woofed that Singapore was “not only doing well, but getting further ahead”. What he did NOT mention was that Singapore and Hong Kong are small city-states. Their economies are relatively small. The US GDP is about $18 trillion while that of Hong Kong (2013) was $274 billion and Singapore’s (2104) was $308 billion. Big difference there.
Moreover, education systems in Singapore and other Asian nations are often referred to as “pressure cookers,” with little emphasis on creativity, curiosity, emotional well-being and leadership.
So, what’s the worry?
All of this “the sky is falling and we will lose the status of empire because of low international math-science test scores” is so much “smoke and mirrors” diversions, and bad inferences, diagnostics and predictions by those with myopic visions and knowledge of history. National success, sustainability and “empire building” (if such is to be desired?) is caused by MUCH more than technical prowess and STEM potentials. Nations rise and fall primarily due to internal factors, such as hard work, basic morality, civility, integrity, self-denial, concern for the oppressed, financial fairness and justice, etc.. Israel fell to Babylon not due to a lack of technical prowess, but due to lack of social and moral justice. Nazi Germany probably had the highest STEM test scores in history at that point, but thankfully Somebody willed that their success would go no further and their attempts at empire failed (their bad fruit outweighed their good fruit, and the Reaper chose to cut down their vine).
Spoken like a true believer, eh, nothing more.
Compared to Duncan and King, I can’t help but think that Devos will be better. These two have systematically blamed everything and everyone, including parents, for their foolish and destructive Politburo-inspired policies.
No, DeVos won’t be better, she will be far, far worse. Duncan and King set the runaway train into motion, she is taking over and heading for a cliff full speed ahead.
“As I have often written before, the international test scores do not predict the future of our economy or anything else. Scores on standardized tests measure family income and income inequality.”
Ay ay ay ay ay!
As I have written before: NO! No standardized test MEASURE ABSOLUTELY NOTHING (quite literally). Not all assessments are measures of something. Standardized tests are an assessment, albeit, a completely invalid one, but they DON’T MEASURE ANYTHING. A correlation, which is what the results of a standardized test can be determined to be with family wealth/income is not a “measurement”. It is an assessment, again invalid due to the invalidity of the results of an invalid process as proven by Noel Wilson (see post below).
But when I state that these invalid standardized tests and results MEASURE ABSOLUTELY NOTHING, I am referring to the fact that, as staunch standardized testing proponent Richard Phelps declares, a standardized test is supposed to “measure the nonobservable”. From his laughingly logically bankrupt defense of said malpractice in the introduction to “Correcting Fallacies About Educational and Psychological Testing”:
“Physical tests, such as those conducted by engineers, can be standardized, of course [why of course of course], but in this volume , we focus on the measurement of latent (i.e., nonobservable) mental, and not physical, traits.” [my addition]
Notice how he is trying to assert by proximity that educational standardized testing and the testing done by engineers are basically the same, in other words a “truly scientific endeavor”. Pretty funny joke of an argument, eh?
So Phelps is saying that we can “measure the nonobservable” with a non-existent measuring device that is not calibrated against a non-existent agreed upon standard unit of learning or an exemplar of said non-existing standard unit.
Where are those property deeds of mine for those wonderful ocean front white and/or black sand beaches I have for sale at Lake of the Ozarks over in Central Missouri???
“No standardized test MEASURE ABSOLUTELY NOTHING (quite literally).” Please change No to A and MEASURE to MEASURES.
Ay ay ay!
For those not familiar with Noel Wilson’s seminal study that outlines all the onto-epistemological errors and falsehoods and psychometrical fudging in the educational standards and standardized testing educational malpractices. It is the most important educational study of the last half a century that has never been refuted or rebutted. Here is a link and my summary:
“Educational Standards and the Problem of Error” found at:
http://epaa.asu.edu/ojs/article/view/577/700
Brief outline of Wilson’s “Educational Standards and the Problem of Error” and some comments of mine.
A description of a quality can only be partially quantified. Quantity is almost always a very small aspect of quality. It is illogical to judge/assess a whole category only by a part of the whole. The assessment is, by definition, lacking in the sense that “assessments are always of multidimensional qualities. To quantify them as unidimensional quantities (numbers or grades) is to perpetuate a fundamental logical error” (per Wilson). The teaching and learning process falls in the logical realm of aesthetics/qualities of human interactions. In attempting to quantify educational standards and standardized testing the descriptive information about said interactions is inadequate, insufficient and inferior to the point of invalidity and unacceptability.
A major epistemological mistake is that we attach, with great importance, the “score” of the student, not only onto the student but also, by extension, the teacher, school and district. Any description of a testing event is only a description of an interaction, that of the student and the testing device at a given time and place. The only correct logical thing that we can attempt to do is to describe that interaction (how accurately or not is a whole other story). That description cannot, by logical thought, be “assigned/attached” to the student as it cannot be a description of the student but the interaction. And this error is probably one of the most egregious “errors” that occur with standardized testing (and even the “grading” of students by a teacher).
Wilson identifies four “frames of reference” each with distinct assumptions (epistemological basis) about the assessment process from which the “assessor” views the interactions of the teaching and learning process: the Judge (think college professor who “knows” the students capabilities and grades them accordingly), the General Frame-think standardized testing that claims to have a “scientific” basis, the Specific Frame-think of learning by objective like computer based learning, getting a correct answer before moving on to the next screen, and the Responsive Frame-think of an apprenticeship in a trade or a medical residency program where the learner interacts with the “teacher” with constant feedback. Each category has its own sources of error and more error in the process is caused when the assessor confuses and conflates the categories.
Wilson elucidates the notion of “error”: “Error is predicated on a notion of perfection; to allocate error is to imply what is without error; to know error it is necessary to determine what is true. And what is true is determined by what we define as true, theoretically by the assumptions of our epistemology, practically by the events and non-events, the discourses and silences, the world of surfaces and their interactions and interpretations; in short, the practices that permeate the field. . . Error is the uncertainty dimension of the statement; error is the band within which chaos reigns, in which anything can happen. Error comprises all of those eventful circumstances which make the assessment statement less than perfectly precise, the measure less than perfectly accurate, the rank order less than perfectly stable, the standard and its measurement less than absolute, and the communication of its truth less than impeccable.”
In other words all the logical errors involved in the process render any conclusions invalid.
The test makers/psychometricians, through all sorts of mathematical machinations attempt to “prove” that these tests (based on standards) are valid-errorless or supposedly at least with minimal error [they aren’t]. Wilson turns the concept of validity on its head and focuses on just how invalid the machinations and the test and results are. He is an advocate for the test taker not the test maker. In doing so he identifies thirteen sources of “error”, any one of which renders the test making/giving/disseminating of results invalid. And a basic logical premise is that once something is shown to be invalid it is just that, invalid, and no amount of “fudging” by the psychometricians/test makers can alleviate that invalidity.
Having shown the invalidity, and therefore the unreliability, of the whole process Wilson concludes, rightly so, that any result/information gleaned from the process is “vain and illusory”. In other words start with an invalidity, end with an invalidity (except by sheer chance every once in a while, like a blind and anosmic squirrel who finds the occasional acorn, a result may be “true”) or to put in more mundane terms crap in-crap out.
And so what does this all mean? I’ll let Wilson have the second to last word: “So what does a test measure in our world? It measures what the person with the power to pay for the test says it measures. And the person who sets the test will name the test what the person who pays for the test wants the test to be named.”
In other words it attempts to measure “’something’ and we can specify some of the ‘errors’ in that ‘something’ but still don’t know [precisely] what the ‘something’ is.” The whole process harms many students as the social rewards for some are not available to others who “don’t make the grade (sic)” Should American public education have the function of sorting and separating students so that some may receive greater benefits than others, especially considering that the sorting and separating devices, educational standards and standardized testing, are so flawed not only in concept but in execution?
My answer is NO!!!!!
One final note with Wilson channeling Foucault and his concept of subjectivization:
“So the mark [grade/test score] becomes part of the story about yourself and with sufficient repetitions becomes true: true because those who know, those in authority, say it is true; true because the society in which you live legitimates this authority; true because your cultural habitus makes it difficult for you to perceive, conceive and integrate those aspects of your experience that contradict the story; true because in acting out your story, which now includes the mark and its meaning, the social truth that created it is confirmed; true because if your mark is high you are consistently rewarded, so that your voice becomes a voice of authority in the power-knowledge discourses that reproduce the structure that helped to produce you; true because if your mark is low your voice becomes muted and confirms your lower position in the social hierarchy; true finally because that success or failure confirms that mark that implicitly predicted the now self-evident consequences. And so the circle is complete.”
In other words students “internalize” what those “marks” (grades/test scores) mean, and since the vast majority of the students have not developed the mental skills to counteract what the “authorities” say, they accept as “natural and normal” that “story/description” of them. Although paradoxical in a sense, the “I’m an “A” student” is almost as harmful as “I’m an ‘F’ student” in hindering students becoming independent, critical and free thinkers. And having independent, critical and free thinkers is a threat to the current socio-economic structure of society.
So Duane, all of your grades of formative and summative assessments in your Spanish classes measure just variance of error and have little to no validity; seek to measure a facet of a psychological construct that does not exist?
Yes, I like Foucault and some of the constructivists and humanists that point out that much of the emphasis we place on testing is misguided and destructive. Yet, “free thinkers” that cannot do basic math and basic language are not going to liberate and enlighten society. Yes, test scores should not be used to measure self-worth, school-worth or anything greater than just the evidence of knowing something at a specific point in time. Yet, to go to the extreme that all testing is “oppressive” and a measure of the “man sticking it to me” to enforce his dominance over me and keep me subjugated to his system….is a great narrative for French existentialists, but is poor pedagogic theory.
Yes, free thinkers and rebels are needed, but they better have the basic knowledge and skills to contribute to society, and not just criticize it because they had low test scores.
Rick,
First we must be careful to distinguish assessments from the concept of grades and assigning grades to a student. Every year I discussed with my students the invalidities of grading and grades and the serious onto-epistemological shortcomings of those malpractices.
And again you have put words in my mouth that I have not said nor implied: “all testing is oppressive” for one or “measure of the man sticking it to me.”
And if you believe that I “just criticize it because [I] had low test scores” you know not what you speak. I criticize these malpractices for their being filled with falsehoods and error and for the harm that is done by the sorting and separating of the students and the inherent Discrimination that is ethically and morally wrong and should be condemned as such.
I agree Duane, and know you are not swallowing Satre’s Nausea to an excessive degree. I just point out that some of your references would lead some to believe that all testing is oppressive and just a tool to manipulate the masses (as some extreme sociologists would suggest). Grace!