This is a 10-minute TED talk by Dr. Yuli Tamir, academic and former minister of education in Israel.
She explains in a direct and lively manner how the PISA standardized testing regime was foisted on the world, destroying children’s imagination, curiosity, and joy of learning.
The fundamental hoax of PISA is the claim that higher test scores will inexorably produce higher economic growth. As she demonstrates, this assertion is false.
If we want children to benefit more from their schooling, we should bend our efforts to reducing poverty. This would seem to be obvious, but it hasn’t slowed the slavish devotion of governments to raising PISA scores.
This is a brilliant presentation. I urge you to watch it.
Diane Ravitch's blog 
This is one of the reasons I absolutely cringe when I hear the word ‘expert’…Always ask myself who is paying this ‘expert’ to say what is being said?
LikeLike
Peskyvera, there are many educators who believe the same about PISA and its destruction of good education. Her point is that higher test scores do not produce economic growth. I made exactly the same point in my 2010 book “The Death and Life of the Great American School System: How Testing and Choice Undermine Education.”
LikeLike
I couldn’t agree more. Well said.
LikeLike
PISA • The Leaning Tower of Baloney
LikeLike
PERFECT!
I wonder how much total $$$$$ has been spent on PISA and other stupid tests?
LikeLiked by 1 person
Enough to help offset the costs of providing an appropriate and free education for all students that doesn’t harm students in the teaching and learning process like the standards and testing malpractice regime does.
“And so what does this all mean? I’ll let Wilson have the second to last word: “So what does a test measure in our world? It measures what the person with the power to pay for the test says it measures. And the person who sets the test will name the test what the person who pays for the test wants the test to be named.” [Actually it doesn’t measure anything. Even Wilson was caught up in the falsehood of the term measure in the teaching and learning process]
In other words it attempts to measure “’something’ and we can specify some of the ‘errors’ in that ‘something’ but still don’t know [precisely] what the ‘something’ is.” The whole process harms many students as the social rewards for some are not available to others who “don’t make the grade (sic)” Should American public education have the function of sorting and separating students so that some may receive greater benefits than others, especially considering that the sorting and separating devices, educational standards and standardized testing, are so flawed not only in concept but in execution?
My answer is NO!!!!!
One final note with Wilson channeling Foucault and his concept of subjectivization:
“So the mark [grade/test score] becomes part of the story about yourself and with sufficient repetitions becomes true: true because those who know, those in authority, say it is true; true because the society in which you live legitimates this authority; true because your cultural habitus makes it difficult for you to perceive, conceive and integrate those aspects of your experience that contradict the story; true because in acting out your story, which now includes the mark and its meaning, the social truth that created it is confirmed; true because if your mark is high you are consistently rewarded, so that your voice becomes a voice of authority in the power-knowledge discourses that reproduce the structure that helped to produce you; true because if your mark is low your voice becomes muted and confirms your lower position in the social hierarchy; true finally because that success or failure confirms that mark that implicitly predicted the now self-evident consequences. And so the circle is complete.”
In other words students “internalize” what those “marks” (grades/test scores) mean, and since the vast majority of the students have not developed the mental skills to counteract what the “authorities” say, they accept as “natural and normal” that “story/description” of them. Although paradoxical in a sense, the “I’m an “A” student” is almost as harmful as “I’m an ‘F’ student” in hindering students becoming independent, critical and free thinkers. And having independent, critical and free thinkers is a threat to the current socio-economic structure of society.”
LikeLike
HAAAAA!!!!
LikeLike
“The fundamental hoax of PISA is the claim that higher test scores will inexorably produce higher economic growth.”
No, that claim is not a fundamental hoax, it is a secondary/tertiary hoax. THE fundamental hoax is that it is claimed to be a valid measuring tool in education. It is not, as PISA suffers all the same invalidities involved in the standards and testing malpractice regime that Noel Wilson (and others) showed us in his seminal 1997 dissertation “Educational Standards and the Problem of Error.”
Brief outline of Wilson’s “Educational Standards and the Problem of Error” and some comments of mine. (updated 6/24/13 per Wilson email)
1. A description of a quality can only be partially quantified. Quantity is almost always a very small aspect of quality. It is illogical to judge/assess a whole category only by a part of the whole. The assessment is, by definition, lacking in the sense that “assessments are always of multidimensional qualities. To quantify them as unidimensional quantities (numbers or grades) is to perpetuate a fundamental logical error” (per Wilson). The teaching and learning process falls in the logical realm of aesthetics/qualities of human interactions. In attempting to quantify educational standards and standardized testing the descriptive information about said interactions is inadequate, insufficient and inferior to the point of invalidity and unacceptability.
A major epistemological mistake is that we attach, with great importance, the “score” of the student, not only onto the student but also, by extension, the teacher, school and district. Any description of a testing event is only a description of an interaction, that of the student and the testing device at a given time and place. The only correct logical thing that we can attempt to do is to describe that interaction (how accurately or not is a whole other story). That description cannot, by logical thought, be “assigned/attached” to the student as it cannot be a description of the student but the interaction. And this error is probably one of the most egregious “errors” that occur with standardized testing (and even the “grading” of students by a teacher).
Wilson identifies four “frames of reference” each with distinct assumptions (epistemological basis) about the assessment process from which the “assessor” views the interactions of the teaching and learning process: the Judge (think college professor who “knows” the students capabilities and grades them accordingly), the General Frame-think standardized testing that claims to have a “scientific” basis, the Specific Frame-think of learning by objective like computer based learning, getting a correct answer before moving on to the next screen, and the Responsive Frame-think of an apprenticeship in a trade or a medical residency program where the learner interacts with the “teacher” with constant feedback. Each category has its own sources of error and more error in the process is caused when the assessor confuses and conflates the categories.
Wilson elucidates the notion of “error”: “Error is predicated on a notion of perfection; to allocate error is to imply what is without error; to know error it is necessary to determine what is true. And what is true is determined by what we define as true, theoretically by the assumptions of our epistemology, practically by the events and non-events, the discourses and silences, the world of surfaces and their interactions and interpretations; in short, the practices that permeate the field. . . Error is the uncertainty dimension of the statement; error is the band within which chaos reigns, in which anything can happen. Error comprises all of those eventful circumstances which make the assessment statement less than perfectly precise, the measure less than perfectly accurate, the rank order less than perfectly stable, the standard and its measurement less than absolute, and the communication of its truth less than impeccable.”
In other words all the logical errors involved in the process render any conclusions invalid.
The test makers/psychometricians, through all sorts of mathematical machinations attempt to “prove” that these tests (based on standards) are valid-errorless or supposedly at least with minimal error [they aren’t]. Wilson turns the concept of validity on its head and focuses on just how invalid the machinations and the test and results are. He is an advocate for the test taker not the test maker. In doing so he identifies thirteen sources of “error”, any one of which renders the test making/giving/disseminating of results invalid. And a basic logical premise is that once something is shown to be invalid it is just that, invalid, and no amount of “fudging” by the psychometricians/test makers can alleviate that invalidity.
Having shown the invalidity, and therefore the unreliability, of the whole process Wilson concludes, rightly so, that any result/information gleaned from the process is “vain and illusory”. In other words start with an invalidity, end with an invalidity (except by sheer chance every once in a while, like a blind and anosmic squirrel who finds the occasional acorn, a result may be “true”) or to put in more mundane terms crap in-crap out.
And so what does this all mean? I’ll let Wilson have the second to last word: “So what does a test measure in our world? It measures what the person with the power to pay for the test says it measures. And the person who sets the test will name the test what the person who pays for the test wants the test to be named.”
In other words it attempts to measure “’something’ and we can specify some of the ‘errors’ in that ‘something’ but still don’t know [precisely] what the ‘something’ is.” The whole process harms many students as the social rewards for some are not available to others who “don’t make the grade (sic)” Should American public education have the function of sorting and separating students so that some may receive greater benefits than others, especially considering that the sorting and separating devices, educational standards and standardized testing, are so flawed not only in concept but in execution?
My answer is NO!!!!!
One final note with Wilson channeling Foucault and his concept of subjectivization:
“So the mark [grade/test score] becomes part of the story about yourself and with sufficient repetitions becomes true: true because those who know, those in authority, say it is true; true because the society in which you live legitimates this authority; true because your cultural habitus makes it difficult for you to perceive, conceive and integrate those aspects of your experience that contradict the story; true because in acting out your story, which now includes the mark and its meaning, the social truth that created it is confirmed; true because if your mark is high you are consistently rewarded, so that your voice becomes a voice of authority in the power-knowledge discourses that reproduce the structure that helped to produce you; true because if your mark is low your voice becomes muted and confirms your lower position in the social hierarchy; true finally because that success or failure confirms that mark that implicitly predicted the now self-evident consequences. And so the circle is complete.”
In other words students “internalize” what those “marks” (grades/test scores) mean, and since the vast majority of the students have not developed the mental skills to counteract what the “authorities” say, they accept as “natural and normal” that “story/description” of them. Although paradoxical in a sense, the “I’m an “A” student” is almost as harmful as “I’m an ‘F’ student” in hindering students becoming independent, critical and free thinkers. And having independent, critical and free thinkers is a threat to the current socio-economic structure of society.
LikeLike
We should send a copy of Prof. Tamir’s talk to Dr. Cardona and the other policy wonks that take up space at the DOE. Prof. Tamir’s talk could also be used to support the rationale for community based schools that can assist students and their families in the schools.
We need to get our public schools off the testing treadmill and away from ever increasing cyber instruction. Now that more than half the students in public schools are poor, we need education that serves, supports and nurtures. We need to abandon data driven policies foisted on the schools by billionaires that seek to privatize our schools. We need to invest in school nurses, social workers and bring back school libraries. Our young people should be the center of what we do in education so they can best serve the nation and themselves. Our youth are more than data points or a line item in some wealthy person’s portfolio.
LikeLike
One of my two favorite German comedians does an occasional routine in which he simultaneously plays three characters who get together in a bar to discuss various things. When he performs it, he sits at a table with a three glasses: wine, beer, and mineral water. The wine drinker is a stuffy, well-educated know-it-all who is usually clueless, the beer drinker a rather course working class, thug who sees the world in black and white, and he’s the water drinker, Erwin Pelzig, an innocent regular guy kind of socially stuck in the 70s but who can always get to the heart of the matter. A number of years ago he did a cd called PISA, which was a series of skits focused on exam questions given by the former to the two latter students. This is the only thing I can post that some might get. The English lesson (NSFW):
LikeLike
Summing up punch line: A frustrated professor is beside himself and can’t understand what they’re saying. Erwin asks if he can speak English. “Of course I can, I speak Oxford English!” To which Hartmut, the beer drinker responds, “Well, we speak Osbourne English.” The professor then asks what they were trying to say, to which Erwin sums up: “Well what I was trying to get across: England is different.”
The point of the whole album is how stupid and arbitrary PISA is. Imagine that, a whole comedy routine on a serious education issue! And enlightening!
LikeLike
Prof. Tamir: “PISA ranks all the participating countries according to the results. And, obviously, everyone wants to be first. This is how education becomes a competition.”
Deming: “Ranking is a farce.”
Deming: “We’re being ruined by competition. What we need is cooperation.”
Deming was with us from 1900 through 1993.
So why is it taking dang so long to “get it?”
Why do we behave as if our Mental Models that spawn competition and adversarialism in one context do not migrate to spawn competition and adverdarialism in other contexts?
LikeLike
Ed,
Thank you for keeping Deming relevant. He certainly is.
LikeLike
Deming was a genius.
LikeLike
I LOVE Deming and his work! The Japanese benefitted from Deming after WWII when rebuilding their economy.
LikeLike
FOISTED: a perfect word for it
LikeLike
The whole PISA testing thing confuses me. What is the “Organisation for Economic Co-operation and Development”? Who cares what they say?
The only positive to the PISA test is that students don’t even seem to know that they are taking it! It isn’t used to compare schools or teachers or students. I guess it is supposed to reveal something about the overall direction an entire nation is going?
The PISA test – aside from all its other flaws – seems to compare countries in the same nonsensical way that benefits charters — it tests a random assortment of students the country wants to test, and the countries that for whatever reason exclude the students who are most likely to do poorly from the testing cohort then look good! The countries that don’t exclude them look worse!
Maybe that’s where some charters CEO got their idea that they could look good by excluding children. I mean, everyone except the clueless journalists who cover education knows this if they have any basic understanding of math. Although it does appear that education journalists lack that basic understanding, as they are absolutely positive that excluding huge numbers of lower-performing students has no connection to high proficiency rates in charters. I wonder if those journalists are just as clueless about the PISA test as well.
At any rate, how did some random test given by some random “organisation” (is it a British group?) ever get taken so seriously? And why? And is it like other “tests” where they are only taken seriously when the results discredit the things the right wing wants to discredit, and then are the results ignored when it discredits things the right wing does not want discredited?
Can anyone start an organization to give a test and then tell countries their education system is good or bad?
LikeLike
I can’t give you the whole history of PISA. It would take more time than I have right now. The US has never performed well on international tests, whether PISA or TIMSS. When the very first intl test was administered in the mid-60s, we came in last. I wrote about this in “The Deasth and life of the Great American School System.” In the 60+ years since then, our economy has outperformed every other nation that had higher scores. The tests do not predict the future. It’s better to have more and better education, but scores on a standardized test of 15-year-olds predict nothing.
Yet whenever the scores come out, the media will write about what a catastrophe they are, “it’s a Sputnik moment,” etc. The reformers use the scores to call for reinventing schools, more school choice, etc. As Duane Swacker would say, “it’s all bull-something.” Yet watch the hue and cry when the scores come out next time. The most affluent kids have the highest scores, the poorest have the lowest. We have more child poverty than the nations we compare ourselves to.
LikeLike
Thank you for this comprehensive reply.
I just don’t remember even hearing about PISA scores until the ed reformers came along. Maybe the media reported it once, but did anyone really care that the US had low PISA scores in the 1970s or 1980s beyond a very small number who researched education?
I can see that the ed reformers glommed onto this and of course their lapdogs in the education media likely decided PISA scores were “very important” because whenever the people who they trust (because the billionaires trust them) tell them something is important they dutifully present that it is very important without having one iota of curiosity about what it means.
Like they do with every narrative the right wing presents to them, if journalists can write a story about how highly respected and completely altruistic people who only care about kids say that these PISA scores are a sign that we need more choice, and add a disclaimer that someone associated with the teachers’ union with an agenda “disagrees”, their job as journalists is done. They have become stenographers of the right wing narrative who believe that including a disclaimer makes their “both sides equal” reporting admirable when it is actually an affront to what journalism is supposed to be. No wonder our democracy is in so much trouble when the “good” education journalists don’t care enough to find out which side is actually presenting an accurate picture of reality. It’s so much easier to just present both sides as equal (except the public school side is pro-union and the ed reform side is altruistic).
LikeLike
Bravo to Yuli Tamir! So clear, concise, and TRUE! This should be required viewing in every school of education, everywhere. Thanks for sharing it, Diane.
LikeLike
PISA has been dubbed the “Olympic Games for education.” Saying that PISA scores affect economic growth is like saying the Olympics affect economic growth. Win all the gold medals you can, it won’t increase your GDP. Duh.
LikeLike
Excellent point, LCT.
LikeLike