Search results for: "PISA"

Yong Zhao writes here about the international test called PISA, which is used to rank, rate and stigmatize entire national systems of education. Its scores are based on a standardized test, of course, which contains the usual flaws of such tests.

His article is called “Two Decades of Havoc.”

Scholars have criticized PISA since it started, but once the global horse race started, there was no slowing it down. PISA now drives every nation to compete for higher scores, in a “race to the top” that very few can win. The critics have been ignored.

It is a stupid metric. Should we really long to be like Estonia? Can all of education be boiled down to questions on a test? What do the results tell us about the future? Nothing, really. When the first international test was given in mathematics in 1964, the U.S. came in last. And over the next 50 years, the U.S. economy surpassed the nations with higher math scores.

 

Tom Loveless has been writing about international assessments for many years. He was quick to blow the whistle on China when the previous international test scores came out, noting that unlike the U.S. and most other nations, China was not testing a cross-section of its students.

In this article on Valerie Strauss’s Answer Sheet blog, Loveless calls out China again for rigging the outcomes to make its students #1.

China’s gains on the tests from 2015 to 2018 were so large as to be incredible, literally not credible.

So the typical change in a nation’s scores is about 10 points. The differences between the 2015 and 2018 Chinese participants are at least six times that amount. The differences are also at least seven times the standard deviation of all interval changes. Highly unusual…

The past PISA scores of Chinese provinces have been called into question (by me and others) because of the culling effect of hukou on the population of 15-year-olds — and for the OECD allowing China to approve which provinces can be tested. In 2009, PISA tests were administered in 12 Chinese provinces, including several rural areas, but only scores from Shanghai were released.


Three years later, the BBC reported, “The Chinese government has so far not allowed the OECD to publish the actual data.” To this day, the data have not been released.
The OECD responded to past criticism by attacking critics and conducting data reviews behind closed doors. A cloud hangs over PISA scores from Chinese provinces. I urge the OECD to release, as soon as possible, the results of any quality checks of 2018 data that have been conducted, along with scores, disaggregated by province, from both the 2015 and 2018 participants.

The OECD allows China to hide data and game the system. This lack of transparency should not stand.

Alan Singer calls out Common Core for the poor showing of US students on PISA. 

Remember all the promises about how Common Core would raise all test scores and close gaps? Nada.

Of course, the deeper issue is that decades of test-and-punish reforms failed, not just Common Core.

it those who pushed these failed policies will not abandon them. They will say—they are saying—that we must double down on failure.

The consensus among governors and policy elites that followed “A Nation at Risk” in 1983 was that common standards, tests, and accountability would lead to high levels of performance (ie, test scores).

They didn’t. They haven’t. They won’t.

Almost four decades later, we can safely say that this theory of reform has failed. Billions of dollars wasted!

Our blog poet reflects on the meaning of the latest international test scores (PISA).

That’s a Moron

When the test hits your eye
Like an old PISA pie
That’s a moron

When the scores make you drool just like a pasta fazool
You’re a moron
When you dance down the street with a test as your beat
You’re insane
When you walk in a dream but you know you’re not dreaming signore
Scuzza me, but you see, back in old Napoli
You’re a moron
A moron, that’s a moron

https://youtu.be/1TWhFmCRdoU

Peter Greene writes regularly for Forbes, where this article appeared.

He explains for the  umpteenth time (as I have done repeatedly) that the U.S. has never led the world on international tests, whether it was PISA, TIMSS, IEA, or any other.

He writes:

The top scores this year come from the usual batch of test takers,including the Chinese, who give the test to students from wealthy provinces.. PISA day is also the one day that some folks hear about Estonia, the tiny nation that somehow has not conquered the world even though their students do well on the PISA.

PISA coverage tends to overlook one major question—why should anyone care about these scores? Where is the research showing a connection between PISA scores and a nation’s economic, political, or global success? What is the conclusion to the statement, “Because they get high PISA scores, the citizens of [insert nation here] enjoy exceptionally good______” ?

Did US companies outsource work to India and China because of their citizens’ PISA scores, or because of low wages and loose regulation? Do we have the world’s most expensive health care system because of mediocre PISA scores? Which politicians have ridden to success on the PISA score platform pony? Are any geopolitical conflicts solved by whipping out the contending countries’ PISA scores for comparison? And is there a shred of evidence that raising PISA scores would improve life for US citizens (spoiler alert: no)?…

There will be discussions of what the PISA scores do or do not prove. Some of that is fair; Common Core and other ed reforms pushed by billionaires and thinky tanks and politicians and a variety of other non-educators were going to turn this all around. They haven’t. This comes as zero surprise to actual educators. It’s just one more data point showing that all the reform heaped on education since A Nation At Risk is not producing the promised results.

Remember when Arne Duncan promoted the “Race to the Top”? Remember when David Coleman and Bill Gates pledged that Common Core would close achievement gaps and raise the lowest-performing students closer to the top-performers? There comes a time when people must be held accountable for their promises.

 

 

Yong Zhao, the brilliant education analyst, writes here about the great PISA illusion. If you have not read any of Zhao’s books, do so now. If you have not heard him speak, google him or invite him to your next big conference. He is insightful, provocative, thoughtful, absolutely delightful! He is a master at making people think and debunking hoaxes.  Please read the entire post to learn how we and the rest of the world have been hoaxed by promoters of fake ideas.

He writes:

PISA is a masterful magician. It has successfully created an illusion of education quality and marketed it to the world. In 2018, 79 countries took part in this magic show out of the belief that this triennial test accurately measures the quality of their education systems, the effectiveness of their teachers, the ability of their students, and the future prosperity of their society.

PISA’s magical power in the education universe stems from its bold claims and successful marketing. It starts by tapping into the universal anxiety about the future. Humans are naturally concerned about the future and have a strong desire to know if tomorrow is better than, or at least as good as, today. Parents want to know if their children will have a good life; politicians want to know if their nations have the people to build a more prosperous economy; the public wants to know if the young will become successful and contributing members of the society.

PISA brilliantly exploits the anxiety and desire of parents, politicians, and the public with three questions (OECD, 1999, p. 7):

  • How well are young adults prepared to meet the challenges of the future?
  • Are they able to analyse, reason and communicate their ideas effectively?
  • Do they have the capacity to continue learning throughout life?

These words begin the document that introduced PISA to the world in 1999 and have been repeated in virtually all PISA reports ever since. The document then states the obvious: “Parents, students, the public and those who run education systems need to know” (OECD, 1999, p. 7). And as can be expected, PISA offers itself as the fortuneteller by claiming that:

PISA assesses the extent to which 15-year-old students, near the end of their compulsory education, have acquired key knowledge and skills that are essential for full participation in modern societies. … The assessment does not just ascertain whether students can reproduce knowledge; it also examines how well students can extrapolate from what they have learned and can apply that knowledge in unfamiliar settings, both in and outside of school. This approach reflects the fact that modern economies reward individuals not for what they know, but for what they can do with what they know. (OECD, 2016, p. 25).

This claim not only offers PISA as a tool to sooth anxiety but also, and perhaps more importantly, makes it the tool for such purpose because it helps to knock out its competitors. As an international education assessment, PISA came late. Prior to PISA, the International Association for the Evaluation of Educational Achievement (IEA) had already been operating international assessments since the 1960s, offering influential programs such as TIMSS and PIRLS. For a start-up to beat the establishment, it must offer something different and better. That’s exactly what PISA promised: a different and better assessment…

However, the claim, the foundation upon which PISA has built its success, has been seriously challenged. First, there is no evidence to justify, let alone prove, the claim that PISA indeed measures skills that are essential for life in modern economies. Second, the claim is an imposition of a monolithic and West-centric view of societies on the rest of the world. Third, the claim distorts the purpose of education.

Made-up Claim

The claim that PISA measures knowledge and skills essential for the modern society or the future world is not based on any empirical evidence. Professor Stefan Hopmann of the University of Vienna writes:

There is no research available that proves this assertion beyond the point that knowing something is always good and knowing more is better. There is not even research showing that PISA covers enough to be representative of the school subjects involved or the general knowledge-base. PISA items are based on the practical reasoning of its researchers and on pre-tests of what works in most or all settings — and not on systematic research on current or future knowledge structures and needs. (Hopmann, 2008, p. 438).

In other words, the claim was just a fantasy, an illusion, entirely made up by the PISA team. But PISA keeps repeating its assertion that measures skills needed for the future. The strategy worked. PISA successfully convinced people through repetition…

Although PISA claims that it does not assess according to national curricula or school knowledge, its results have been interpreted as a valid measure of the quality of educational systems. But the view of education promoted by PISA is a distorted and extremely narrow one (Berliner, 2011; Sjøberg, 2015; Uljens, 2007). PISA treats economic growth and competitiveness as the sole purpose of education. Thus it only assesses subjects — reading, math, science, financial literacy, and problem solving — that are generally viewed as important for boosting competitiveness in the global economy driven by science and technology. PISA shows little interest in other subjects that have occupied the curricula of many countries such as the humanities, arts and music, physical education, social sciences, world languages, history, and geography (Sjøberg, 2015).

While preparing children for economic participation is certainly part of the responsibility of educational institutions, it cannot and should not be the only responsibility (Labaree, 1997; Sjøberg, 2015; Zhao, 2014, 2016). The purpose of education in many countries includes a lot more than preparing economic beings. Citizenship, solidarity, equity, curiosity and engagement, compassion, empathy, curiosity, cultural values, physical and mental health, and many others are some of the frequently mentioned purposes in national education goal states. But these aspects of purpose of education “are often forgotten or ignored when discussions about the quality of the school is based on PISA scores and rankings” (Sjøberg, 2015, p. 113).

Zhao presents a devastating critique of the validity of PISA. It is a must read.

Politico Morning Education reports:

 

U.S. SCORES IN READING, MATHEMATICS AND SCIENCE LITERACY REMAINED ESSENTIALLY FLAT FROM 2015 in the latest Program for International Student Assessment results, but U.S. rankings improved because other education systems worsened.

— The 2018 PISA results showed U.S. average scores in reading and science literacy were higher than the average of about three dozen mostly industrialized countries making up the Organization for Economic Cooperation and Development, which develops and coordinates the assessment. But U.S. average math scores were lower than the OECD average.

— PISA, an international assessment administered every three years, measures 15-year-old students’ literacy in the three disciplines and is designed to provide a global view of U.S. students’ performance compared to their peers in nearly 80 education systems.

— “If I communicated nothing, I hope I communicated that we are struggling in math in comparison to our competitors around the world,” Peggy G. Carr, the associate commissioner of assessments for the National Center for Education Statistics, told reporters in a call before the results were released. Nicole Gaudiano has more.

 

The PISA results were released, and they put the test-and-punish reforms of the past two decades in a harsh light. Billions have been spent on testing and spurious teacher evaluations.

Dana Goldstein writes in the New York Times:

The performance of American teenagers in reading and math has been stagnant since 2000, according to the latest results of a rigorous international exam, despite a decades-long effort to raise standards and help students compete with peers across the globe.

And the achievement gap in reading between high and low performers is widening. Although the top quarter of American students have improved their performance on the exam since 2012, the bottom 10th percentile lost ground, according to an analysis by the National Center for Education Statistics, a federal agency.

If you recall, the Disrupters claimed that their method would both “Race to the Top” and “close achievement gaps.”

Their strategies did neither. Time for a change.

This is an ironic story. There is no one and no institution that has done more to set off an international test score competition than Andreas Schleicher of the OECD, which administers the periodic international tests called PISA, the Programme in International Student Assessment. Every nation wants to be first. Every nation waits anxiously to see whether its test scores in reading, mathematics, and science went up or down. In 2010, when the 2009 PISA scores were released, Arne Duncan and Barack Obama declared that the U.S. was facing another “Sputnik moment,” and it was time to crack down. Others wrung their hands and wondered how we could toughen up to compete with Shanghai.

Yet Scheicher testified recently to a committee of the House of Commons that arts education may be more valuable than the academic skills that are tested.

The arts could become more important for young people than maths in the future, according to a leading education expert.

Researcher Andreas Schleicher, who leads the Programme for International Student Assessment at the intergovernmental economic organisation OECD, told a House of Commons inquiry that he believed young people could benefit more from the skills gained through creativity than test-based learning.

He was giving evidence to the Education Select Committee as part of an ongoing inquiry into the fourth industrial revolution – the influence of technologies such as robotics and artificial intelligence on society.

Schleicher, who is widely regarded as one of the world’s leading educational thinkers, said: “I would say, in the fourth industrial revolution, arts may become more important than maths.”

“We talk about ‘soft skills’ often as social and emotional skills, and hard skills as about science and maths, but it might be the opposite,” he said, suggesting that science and maths may become ‘softer’ in future when the need for them decreases due to technology, and the ‘hard skills’ will be “your curiosity, your leadership, your persistence and your resilience”.

His comments come amid ongoing concerns about the narrowing of the education system in the UK to exclude creativity and prioritise academic subjects.

Campaigners argue that this is prohibiting many young people from pursuing creative careers. However, Schleicher said that too narrow a curriculum could also make young people less prepared for the demands of the future.

 

PISA—the international test, the Program in International Student Assessment—has set off an insane competition among nations to lift their ranking. Only one country can be #1, and the rankings have political consequences. Rich countries always get higher scores than poor ones. Nations with less poverty get higher scores than those with more poverty.

The US typically ranks in the middle, not because it is a poor country but because it has very high rates of child poverty. But the news media always report the results like a horse race and blame the schools because we are not number one. We have never been number one on international assessments because of the 20-25% of our children who live in poverty. Yet neither the federal nor state governments have adopted a goal of reducing child poverty.

The media simply refuse to acknowledge that the tests tell us that poverty matters. Instead, they produce raw meat for demagogues with simple solutions, like Michelle Rhee, Campbell Brown, and Arne Duncan, now DeVos. When the PISA results are released, it is another opportunity to moan about “a Sputnik moment” and dreams of becoming more like South Korea or Shanghai.

Why don’t the media or the politicians say it is time to emulate Finland, which has high rankings, low child poverty, and no standardized testing?

William Stewart of the British TES (Times Educational Supplement) reports that teachers are feeling anxiety over national rankings. 

Why are the nations of the world bothering to participate? Maybe it is a matter of national pride, even though most are doomed to “fail.”

Wouldn’t it be wonderful if nations opted out?