A letter written by State University of Albany’s Heinz Dieter Meyer and educator Katie Zahedi protested the negative effects of PISA on education goals because of its emphasis on standardized tests and international competition. The letter has been translated into many languages and collected hundreds of signatures from scholars and educators around the world.
The letter was addressed to Andreas Schleicher of OECD, who is director of PISA.
If you wish to sign the letter, it is here.
Dr. Schleicher responded promptly to the letter, saying it was based on “false claims,” and that it has not caused “short term fixes,” as a way for nations to raise their national rankings. Of course, some Americans would say that No Child Left Behind and Race to the Top was driven by the goal of international competition. On this ground, both programs were failures, leading to more testing, more measures to rank and rate students, teachers, and schools.
Since it is impossible to get a unified response from the many who have signed it, Dr. Meyer has invited signatories to submit their own responses, which he will make available on a website.
For now, the best available response to Dr. Schleicher is an article (part of a series) about how PISA has harmed education reform in the nations of East Asia, the putative “winners” of the PISA contest. Zhao calls his series “”How does PISA Put the World at Risk?”
Zhao says if he were a conspiracy theorist, he would think that PISA is a western plot to keep China trapped in an antiquated system, and unable to try the education reforms that would usher in a new era of creativity and entrepreneurship.
“He writes:
“Such a citizenry is urgently needed for China’s successful transition from a labor-intensive economy to one that relies on innovation, a transition China must make for its future development. The Chinese exam-oriented education has long been recognized as the culprit for limiting China’s capacity for producing creative and diverse talents. Just as China’s education reforms began to touch the core of its traditional education—the gaokao or College Entrance Exam and the wide use of testing at all levels of education, PISA announced that the Chinese education is the best in the world. And the exam system, including the gaokao, is glorified as a major contributor to China’s success, making it difficult for the Chinese to continue the battle against testing.”
He writes further:
“If I expanded the conspiracy theory, I could say that PISA is a plot to disrupt all Eastern Asian countries’ serious efforts to develop an education system that cultivates confident, creative, diverse, and happy students. For example, PISA “played a role in the decision to reverse, at least in part, the yutori reform launched at the beginning of the decade,” writes a 2011 OECD document[2]. Yutori kyoiku (roughly translated “relaxed education” or education with some freedom) was a major education reform movement started in the 1980s in Japan. “The yutori reform was based on an emerging consensus that the school system was too rigid and that a new approach was needed to encourage creativity,” observes the OECD document[3]. The major changes included reduction in school days and a 30% cut in the school curriculum. “In addition, the government relaxed grading practices and introduced “integrated learning classes” without textbooks in an effort to help students think independently and reduce the importance of rote learning” [4]. The changes were announced in 1998 and implemented in 2002. “The ultimate desire was to instill in students ‘a zest for learning.’”[5]
“In 2003, Japan’s PISA rankings fell, resulting in a public panic over Japan’s decline in international academic standing. Opponents of the yutori reform seized the moment and blamed the reform for the decline. In response, Japan decided to water down the previous reforms with increase in required topics in standard academic subjects, increase time devoted to these subjects, and introducing national standardized testing in math and Japanese for the first time in 2007.
“Putting someone on a pedestal is an effective way to ensure he does not veer far from his previous behaviors because any deviation could tarnish the bestowed honor. The Chinese call such actions pengsha or “killing with flattery.” Pengsha derives from a story recorded almost 2,000 years ago: A nobleman rides on a beautiful horse and wins great praises from admiring onlookers. Enjoying the flattery, the nobleman keeps on riding till the horse dies from exhaustion.
“PISA has certainly successfully put a number of East Asian education systems on a pedestal and thus constrained their ability and desire to make drastic changes. But they need drastic changes if they wish to truly cultivate the kind of talents needed to become innovative societies that rival the West because the authoritarian East Asian education model leaves little room for creative and unorthodox individuals to pursue their passion, question the authority, and develop their strengths, although it is extremely effective in homogenizing individuals, enforcing compliance, and hence producing great test scores.
“PISA’s claims about progress East Asian education systems have made over the years can further convince them to keep riding their horses. It gives them the illusion that they are moving forward, in the right direction, because their PISA rankings keep going up. But in reality, East Asian education systems have never “risen,” as PISA often claims. They have always been great test takers. Singapore, Korea, Japan, and Hong Kong scored extremely well on international tests succ as TIMSS prior to the birth of PISA. Shanghai did not participate in these studies but if it did, it would have scored well.
Ultimately, Yong Zhao abandons the conspiracy theory because PISA does even more harm to the western nations than to the east.
He concludes:
“By attracting poor, developing countries into a senseless academic race, PISA wastes precious resources of these countries. While the 182,000 euros (about US$250,000) participation fee[6] and millions of dollars implementation costs may not be much for developed nations, it can be a huge burden for developing countries. More important, the money can buy a lot more meaningful education resources—pencils, for example—than humiliating PISA rankings or policy advice that cannot be implemented.
“PISA is a good servant but a bad master,” wrote Finnish education scholar Pasi Sahlberg, author of the Finnish Lesson: What Can the World Learn from Educational Change in Finland. Pasi is, as always, wise and generous, but in my mind, PISA is a servant that has turned into a bad master, perhaps by design. As it commands the world to race to fix the old paradigm and forgo opportunities to invent a new one, it puts the entire world at risk.”
My hope is that thousands and thousands of educators add their names to the letter of protest against the false values promoted by PISA.
Diane Ravitch's blog 
I think this whole PISA debate is a distraction. No doubt, the results have been misused and misinterpreted. But if PISA were abandoned, there would be something else. And I if PISA were abandoned, we would still have the same issues to deal with – evaluation, high-stakes testing, privatization, ….
We need to pick our battles – ones that can really affect what actually goes on in classrooms, ones that teachers and parents can grab onto, ones that can engage voters. Large policy issues like how evil PISA.is just one that is going to hunt where it counts.
It seems like we’re heading for another policy debate instead of focusing needed energy on what we really want for education.
Note that this in no way takes a position on PISA.
Peter,
“Note that this in no way takes a position on PISA.”
I’m not sure what you are getting at with that final sentence. Please explain.
TIA (Thanks in advance!)
PISA has all the same epistemological and ontological errors that plague all standardized testing and therefore any conclusions drawn are COMPLETELY INVALID* and is at best a bunch of mental masturbation using excrement of bovine origin. Not pretty, eh!
*“Educational Standards and the Problem of Error” found at:
http://epaa.asu.edu/ojs/article/view/577/700
Brief outline of Wilson’s “Educational Standards and the Problem of Error” and some comments of mine. (updated 6/24/13 per Wilson email)
1. A quality cannot be quantified. Quantity is a sub-category of quality. It is illogical to judge/assess a whole category by only a part (sub-category) of the whole. The assessment is, by definition, lacking in the sense that “assessments are always of multidimensional qualities. To quantify them as one dimensional quantities (numbers or grades) is to perpetuate a fundamental logical error” (per Wilson). The teaching and learning process falls in the logical realm of aesthetics/qualities of human interactions. In attempting to quantify educational standards and standardized testing we are lacking much information about said interactions.
2. A major epistemological mistake is that we attach, with great importance, the “score” of the student, not only onto the student but also, by extension, the teacher, school and district. Any description of a testing event is only a description of an interaction, that of the student and the testing device at a given time and place. The only correct logical thing that we can attempt to do is to describe that interaction (how accurately or not is a whole other story). That description cannot, by logical thought, be “assigned/attached” to the student as it cannot be a description of the student but the interaction. And this error is probably one of the most egregious “errors” that occur with standardized testing (and even the “grading” of students by a teacher).
3. Wilson identifies four “frames of reference” each with distinct assumptions (epistemological basis) about the assessment process from which the “assessor” views the interactions of the teaching and learning process: the Judge (think college professor who “knows” the students capabilities and grades them accordingly), the General Frame-think standardized testing that claims to have a “scientific” basis, the Specific Frame-think of learning by objective like computer based learning, getting a correct answer before moving on to the next screen, and the Responsive Frame-think of an apprenticeship in a trade or a medical residency program where the learner interacts with the “teacher” with constant feedback. Each category has its own sources of error and more error in the process is caused when the assessor confuses and conflates the categories.
4. Wilson elucidates the notion of “error”: “Error is predicated on a notion of perfection; to allocate error is to imply what is without error; to know error it is necessary to determine what is true. And what is true is determined by what we define as true, theoretically by the assumptions of our epistemology, practically by the events and non-events, the discourses and silences, the world of surfaces and their interactions and interpretations; in short, the practices that permeate the field. . . Error is the uncertainty dimension of the statement; error is the band within which chaos reigns, in which anything can happen. Error comprises all of those eventful circumstances which make the assessment statement less than perfectly precise, the measure less than perfectly accurate, the rank order less than perfectly stable, the standard and its measurement less than absolute, and the communication of its truth less than impeccable.”
In other word all the logical errors involved in the process render any conclusions invalid.
5. The test makers/psychometricians, through all sorts of mathematical machinations attempt to “prove” that these tests (based on standards) are valid-errorless or supposedly at least with minimal error [they aren’t]. Wilson turns the concept of validity on its head and focuses on just how invalid the machinations and the test and results are. He is an advocate for the test taker not the test maker. In doing so he identifies thirteen sources of “error”, any one of which renders the test making/giving/disseminating of results invalid. As a basic logical premise is that once something is shown to be invalid it is just that, invalid, and no amount of “fudging” by the psychometricians/test makers can alleviate that invalidity.
6. Having shown the invalidity, and therefore the unreliability, of the whole process Wilson concludes, rightly so, that any result/information gleaned from the process is “vain and illusory”. In other words start with an invalidity, end with an invalidity (except by sheer chance every once in a while, like a blind and anosmic squirrel who finds the occasional acorn, a result may be “true”) or to put in more mundane terms crap in-crap out.
7. And so what does this all mean? I’ll let Wilson have the second to last word: “So what does a test measure in our world? It measures what the person with the power to pay for the test says it measures. And the person who sets the test will name the test what the person who pays for the test wants the test to be named.”
In other words it measures “’something’ and we can specify some of the ‘errors’ in that ‘something’ but still don’t know [precisely] what the ‘something’ is.” The whole process harms many students as the social rewards for some are not available to others who “don’t make the grade (sic)” Should American public education have the function of sorting and separating students so that some may receive greater benefits than others, especially considering that the sorting and separating devices, educational standards and standardized testing, are so flawed not only in concept but in execution?
My answer is NO!!!!!
One final note with Wilson channeling Foucault and his concept of subjectivization:
“So the mark [grade/test score] becomes part of the story about yourself and with sufficient repetitions becomes true: true because those who know, those in authority, say it is true; true because the society in which you live legitimates this authority; true because your cultural habitus makes it difficult for you to perceive, conceive and integrate those aspects of your experience that contradict the story; true because in acting out your story, which now includes the mark and its meaning, the social truth that created it is confirmed; true because if your mark is high you are consistently rewarded, so that your voice becomes a voice of authority in the power-knowledge discourses that reproduce the structure that helped to produce you; true because if your mark is low your voice becomes muted and confirms your lower position in the social hierarchy; true finally because that success or failure confirms that mark that implicitly predicted the now self-evident consequences. And so the circle is complete.”
In other words students “internalize” what those “marks” (grades/test scores) mean, and since the vast majority of the students have not developed the mental skills to counteract what the “authorities” say, they accept as “natural and normal” that “story/description” of them. Although paradoxical in a sense, the “I’m an “A” student” is almost as harmful as “I’m an ‘F’ student” in hindering students becoming independent, critical and free thinkers. And having independent, critical and free thinkers is a threat to the current socio-economic structure of society.
I signed the letter. It is worth noting that the Brookings Institute has become a hospitality center for economists who are beating the drum for education in the service of global competition.
In case you missed this reprieve of “a Nation at Risk,” here it is with Eric Hanusheck at it again…over 500 citations with his name in Google Scholar promoting economic criteria for judging the efficacy of education here and internationally.
Hanushek, E. A., Peterson, P. E., & Woessmann, L. (2013). Endangering Prosperity: A Global View of the American School. Brookings Institution Press.
I signed the letter, too. Andreas Schleicher may paint his detractors as whiners from the west (mainly, the US), but he clearly overestimates the meaning of PISA test scores to assess the quality and progress of educational system in each country. OECD’s gaming system counts those who are culturally and socio-economically advantaged and dismisses those who are not. That’s how the table places some Asian countries like Japan, China, Singapore, and South Korea in a relatively high position among all countries in the table.
I agree with Zhao on the grand scheme of OECD’s gaming system that attempts to frustrate Asian countries from making a productive education system. As he mentions, Yutori (relaxed) education was an interesting idea proposed by an incompetent Japan’s MEXT. Problems? They made in rash for implementing reduced-class hour curriculum–1) without field testing; 2) without an input from experienced teachers/educators; and 3) without even thinking how to fix mandatory the pre-existing entrance exam system. Japanese critics of relaxed education also bought into the myth of PISA table. They lashed out the MEXT after the release of 2003 PISA test scores. They conveniently ignored the fact that 1) “yutori” was not in full effect(it was implemented in the same year during the PISA survey was conducted; 2) Japan’s PISA ranking was already in decline since the late 1990s; and 3) several countries joined as a newcomer since the year 2003. It was MEXT’s sloppy curriculum decision making, which could highly be attributed to their institutional insularity–not PISA test scores that cost the effectiveness of “yutori.”
Funny Schleicher seems to believe that the international score table can evaluate overall quality of education system in each country. Japan is currently in Top 5 among OECD nations for PISA 2009 and 2012, but I haven’t heard anyone saying Japanese education is making progress today. Not a single one. What I usually hear most is mental anguish of students experiencing mundane, rote learning, bullying, harassment, and/or excessive disciplinary action by some unconscionable teachers that brought physical harm to students–or even worse, suicide. Contrary to positive international recognition, Japan is currently dealing with cultural insularity from the world outside the country: an increasing number of students who tend to stay inside the country after K-12. Also, there are very few foreign-born students who go to Japanese public schools. Why? Because many Japanese schools are hesitant to accept the students of foreign workers — partly, due to insufficient second language program, ignorance of racial consciousness, and concern over bullying or harassment (a perfect excuse for channeling cultural homogenous myth to boost PISA scores, huh?) Even the number of international students coming to Japanese university has been in decline in the last few years. I don’t see any sign of progress in education system of my home country through the lens of international scores table.
Sorry, Mr. Schleicher. I call it crank.
How do we educators sign this letter?
I added a link that enables you to sign the anti-PISA letter
Diane,
Thank you!
Bill
Validity of a test isn’t the same as reliability. Reliability means it consistently measures whatever it is measuring, and all these standardized tests can do this and the testmakers’ protests that their tests are “valid” when all they’ve shown is that they correlate to similar tests does not make them valid.. Validity means it measures what it is SUPPOSED to measure. To do this, you’d need to find out what actually matters and see whether the test scores are consistent with what actually matters. Not clear how to do this, and pretty clear that this hasn’t been done. And then taking these individual test scores and averaging them to the ninth decimal place produces a spurious sense of precision.
and pretty clear that this hasn’t been done
precisely
Signed. Thank you, Diane, for posting this. Hope you are recovering well.