This is the most important post you will read this month or maybe even this year. It refutes the basis of American education policy.
This is major study of the relationship between scores on PISA and economic growth. It demonstrates that there is none.
It was written by Hikaru Komatsu (Associate Professor at National Taiwan University) and Jeremy Rappleye (Associate Professor at Kyoto University, Graduate School of Education) for the Network for International Policies and Cooperation in Education and Training. The authors criticize the work of Hoover economist Eric Hanushek and demonstrate how his theories of human capital development were widely adopted by American and European organizations and became the convention wisdom.
Komatsu and Rappleye demonstrate the flaws in Hanushek’s theories, which have led to unprecedented emphasis on improving standardized test scores in many nations.
They begin by reviewing a paper published by the European Commission, based on Hanushek’s human capital theories. Open the link to see the graphs.
The EC report was written by Eric Hanushek (Hoover Institute, a think tank on the campus of Stanford University) and Ludger Woessmann (University of Munich, ifo Center for Economics of Education). It laid out the same findings, methods and arguments that can be found in a range of publications in the United States dating back to the early 2000s (e.g., Hanushek & Kimiko, 2000), and reaching back even – with a bit more historical awareness – to the heady Anglo-American neo-liberalism of Reagan and Thatcher in the 1980s (e.g., Hanushek, 1981, Hanushek, 1986). These claims were articulated strongly in a 2013 book by the same authors, published by the Brookings Institute and intended to reach US policymakers, entitled Endangering Prosperity: A Global Look at the American School. These same findings were also publicized in major reports by the World Bank (2007) and the OECD (2010, 2015), both of which commissioned Hanushek and Woessmann to write their findings into development policy. The World Bank would later officially adopt the model as the underpinning logic of Learning for All (2011) (see Auld, Morris, and Rappleye, 2019), while the OECD’s 2010 report entitled The High Cost of Low Educational Performance – The Long-Run Impacts of Improving PISA Outcomes would be virtually transferred carbon copy into the EC’s 2019 report. That is, the EC 2019 Report claims that an aggressive, focused 15-20 year reform push to raise scores by 25-points would “add €71 trillion to EU GDP over the status quo” and which “amounts to an aggregate EU gain of almost 3 times current levels of GDP and an average GDP that is seven percent higher for the remainder of the century”. Based on the Hanushek and Woessmann numbers, Andreas Schleicher enticed European leaders with precisely that same narrative in 2010, as shown here in a slide from his presentation below (Slide 34 in the original presentation). Schleicher claimed that a PISA-improvement reform add 30% of the current GDP in 2100, which makes the total economic value of this reform is equivalent to 340% of the current GDP – the exact value shown in Figure 2…
For quite some time, we and others (here, here, here, and here) have pointed out that the Hanushek and Woessmann “findings” are deeply flawed. Our work includes a number of published papers, newspaper articles, and blogs published since 2016. We have tried to call attention to this situation in two previous NORRAG blog pieces here and here. Our argument in the main 2017 paper was simple. Hanushek and Woessmann used a relationship between students’ performance in international tests and economic growth for estimating the economic value of improving 25 points of PISA scores. However, Hanushek and Woessmann surprisingly compared students’ performance for a given period and economic growth for the same period. However, as it logically takes several decades for that cohort of students to occupy a major portion of workforce and then contribute to economic growth. We logically compared students’ performance for a given period and economic growth for a subsequent period. Surprisingly, in doing so we discovered virtually no relationship between them, casting strong doubts on the purportedly strong causal claims (Komatsu & Rappleye, 2017). While we find it disheartening that there has been no response to our work, it is far more disappointing that find that now the EC have turned to Hanushek and Woessmann, paying them hefty consultancy fees to write policy recommendations for Europe. We wonder aloud: Why does the EC Directorate for Education, Youth, Sport, and Culture need to turn to American think tanks to generate new policy ideas?…
Returning again to the larger picture, it seems that now the EU and OECD, alongside the World Bank, OECD and often highly influential figures in UNESCO, are now utilizing the same Hanushek and Woessmann Knowledge Capital claims. What makes this ‘Western consensus’ so alarming – at least to us – is not simply that education and economics are being so tightly coupled or that PISA is being embedded deeply into policymaking goals through these works. It is, instead, that so many leading minds in the West seem unable or unwilling to think differently.
A decade ago, when I wrote The Death and Life of the Great American School System: How Testing and Choice Are Undermining Education, I quoted a study by Keith Baker, a statistician who worked for many years in the U.S. Department of Education. Baker pointed out that the U.S. had placed last in the first international assessment in 1964, yet over the next half-century had outperformed the eleven nations with higher scores. He concluded then that test scores do not predict economic growth or anything else. Every time the results of a new international assessment are released, whoever is in charge says that the performance of the U.S. students is horrible, shameful, alarming, and proclaims “a new Sputnik moment.” And every time I point out that the U.S. has never been number one on international assessments and that these scores are meaningless. But the press reports the lamentations without contradiction anyway.
Gosh, that’s a shock. Never saw that coming.
BTW, what borders on insanity?
Canada and Mexico.
Thanks I needed that 😀
LMAO!
BTW, would it matter if standardized test scores did have a positive effect on economic growth? The harms of standardized testing are well documented. Aren’t we the ones saying we can’t sacrifice our children for the economy? The purpose of education is to produce thinking, productive, well-rounded citizens of a(n alleged) democratic republic, not drive the economy.
Thank you. I needed that.
A secular Amen to that, Dienne!
Check! There are many inhumane practices that have a positive effect on economic growth.
I remember a study by Harvard economist Roland Fryer that demonstrated that teachers would work harder and get higher test scores if they had negative incentives, for example, if they were promised a bonus of $4,000 but then lost it because of low scores. It made me wonder about other negative incentives. For example, if the scores don’t go up, we cut off a finger. That might produce higher scores, but at what cost?
I am remembering the Atlanta teacher ‘test-score cheating’ scandal — the negative incentive there was along the lines of “Get Higher Scores Or Lose Your Job”
ciede-
If the judge in the case rendered the same decision today, in the climate of BLM, he would be forced into retirement. Good riddance to right wing judges.
I think Fryer must have been putting his theory into practice at Harvard, thinking his female students would work harder if they were sexually harassed.
SDP, good one!
Thank you to Hikaru Komatsu (Associate Professor at National Taiwan University) and Jeremy Rappleye (Associate Professor at Kyoto University, Graduate School of Education) for the Network for International Policies and Cooperation in Education and Training. And thank you Diane for exposing again the absurdity of linking test scores to economic outcomes.
Hanushek and Woessmann (with Paul Peterson and other economists) have been peddling snake oil about the significance of test scores in many venues. They invented a fiction that standard deviations in test scores (math or ELA) should be interpreted as days (months, years) of learning gained or lost. You are led to believe that only test scores in these two subjects matter and that students only study these two subjects.
“We logically compared students’ performance for a given period and economic growth for a subsequent period. Surprisingly, in doing so we discovered virtually no relationship between them, casting strong doubts on the purportedly strong causal claims (Komatsu & Rappleye, 2017).”
We have only been operating under false assumptions for the past forty years! We can add test scores and economic growth to the long list of erroneous and misleading assertions that have plagued public schools for years. We can throw it all in the privatization dustbin along with VAM, test and punish, charters, vouchers, “failing public schools,” personalized learning and so many other falsehoods that seek to rob public education in order to make it collapse under the weight of all the lies from economists and manipulations from the billionaire class.
We have been operating under false assumptions for the past forty years!
Precisely. Gates paid Achieve to be the major propaganda organ for the testing regime (which would sell a lot of computer software), and the BIG ARGUMENT propagated by Achieve and featured, for years, on top of the homepage of its website, was that the United States was facing economic ruin because its PISA scores were lower than those of other countries. Of course, there was no actual strong correlation between PISA scores and economic outcomes, and if you weighted the scores by the socioeconomic status of the students taking the tests, the US was not behind other countries.
It’s always been poverty that was the issue. Gates used Achieve and lots of other astroturf organizations to sell the nation on this testing scam (and on the Common Core) so that he could sell more product–it was all part of a business plan to make K-12 and then college education all about using computers to generate “data Numerology and snake oil.
Diane is right to point to the importance of this article. It’s yet another Emperor’s New Clothes moment for Ed Deform, and it’s important because shows that Ed Deform’s foundational argument was total bs.
Plus, the grandaddy of all that, “A Nation at Risk,” was just another in the salvo of shiny objects released to take eyes off how our fearless leaders were actually dealing with automation/ rise of E Asian competition.
Most of the field of economics has been operating under false assumptions for a lot longer than forty years.
Nearly every assumption made by mainstream (neoclassical) economics is false.
“We find it disheartening that there has been no response to our work” and “he EC [has] turned to Hanushek and Woessmann, paying them hefty consultancy fees to write policy recommendations for Europe”
It doesn’t matter what the truth is–what the actual science says. The Hanushek magic elixir (improve test scores!) fits preexisting right-wing biases, so it HAS to be true. The same is true with the notion that external, extrinsic punishment and reward systems improve motivation for cognitive tasks. And the same is true of the idea that increases in the minimum wage don’t help poor people because they just drive up prices.
As the right, here, has shown abundantly during the IQ45 era, it is willing to believe anything that fits with an ideology that justifies a few fat cats at the top and a whole lot of unempowered worker bees at the bottom. It’s not our ZIP code, it’s not our wealth, it’s our innate superiority–we’re a bunch of stable geniuses. Those poor kids just need to be tested a lot more and subjected to harsh discipline (and not just kids, btw–that’s why we need federal secret police). Law n ordure, not living wages, for the poor!
Standardized testing can be used to justify existing economic inequity. That’s why the right wingers love standardized tests. They and their children are where they are because of merit, not because of wealth. Of course they embrace pseduoscience that purports to prove their demented, self-serving worldview. Here, the truth about the standardized testing we’re doing in the United States today: https://bobshepherdonline.wordpress.com/2020/03/19/why-we-need-to-end-high-stakes-standardized-testing-now/
Thanks for something to read.
It’s quite brief. And devastating to the proponents of standardized testing.
This study raises the important question: why do we require standardized tests since they don’t predict anything that matters.
As usual, Diane, you go directly to the heart of the issue. We give them because, as you have pointed out in book after book, they are extremely good predictors of ZIP code and so prop up the privilege of the children of the wealthy.
Hello Bob,
Yes, that is one answer. But on a deeper level, our whole Western consciousness is geared towards the rational, scientific way of knowing. That perspective places emphasis on reason, measurement, certainty and cuts out the other ways of knowing – imagination, knowing through the body and senses and having direct, subjective experience. It’s pretty much impossible to quantify, predict and place a “value” on this way of being in the world. And perhaps more difficult to control and make money from. 🙂
I am searching for words to express how much I agree with this. Profound, Mamie, and beautifully said. We need much wider (and more rational) notions of what’s rational than we have inherited from the Enlightenment, which attempted to split us down the middle–not exactly a rational thing to do to an ORGANISM.
Don’t know if you’ve seen this, Mamie, but I bet you would love it: https://www.youtube.com/watch?v=dIFsZ9uTrpE
While it is not Tolstoy’s ‘War and Peace’ or Adam Smith’s ‘An Inquiry into the Nature and Causes of the Wealth of Nations’ I find what I haven’t read to be more than brief.
First, I read the blog (brief), then I created a new PISA folder, and saved a file copy of the blog for future reference and citation. Then I downloaded the following two documents.
1. The economic benefits of improving educational achievement in the European Union An update and extension executive summary NC0119648ENN.en.pdf (9 pages brief)
2. The economic benefits of improving educational achievement in the European Union An update and extension NC0119647ENN.en-1.pdf (60 pages, not read yet.)
And since I am critically reading the NORRAG blog post, “TINA comes to European Education? The European Commission, PISA, PIAAC, and American-Style Knowledge Capital Theory by Hikaru Komatsu and Jeremy Rappleye”
Which means I will follow and read every link in the blog, and possibly every link found in the links.
And because I am not digging through and translating PISA results for my book yet, completion of the critical reading is not an immediate requirement. However, as I find additional information related to the general point, I will add it to the ‘PISA – TINA – ECON’ folder.
HAAAA!!!! Good. In my defense, the post IS brief, as you note, and I was simply hoping to encourage folks actually to read it.
The question of why no correlation was found between the test results and the economic outcomes is an interesting one. However, those who start with the conviction that the former determines the latter, those for whom that’s axiomatic, are likely to stumble on some complex, tortured, fallacious argument in support of it. That’s what folks at, for example, the Fordham Institute for Securing Big Paychecks for the Officers of the Forham Institute do for those checks, for example.
Bob,
I looked at your page and saved the link to read later. Thanks.
Hi Bob,
Ok, where to begin? Thanks for that video. It was very thought-provoking. Yes, Heidegger was one of those who was in opposition to the Cartesian world view. Actually, there is a whole kind of undercurrent in our history which contrasts with the rational. A long conversation to be had there to be sure. I guess the most interesting thing was the discussion of our technological/informational world and the question of making meaning of our lives and experience. I also think a lot about the extent to which technology is viewed as a kind of panacea to all our problems. They mention this in the video. We are meaning making beings and one of the problems of our time is how to make meaning in this technological world which, in a certain sense, helps us to forget that this is integral to us as human beings. Another big discussion to be had here. I can see some cracks in our rational and neat and tidy world-view right now. Some of them have to do with our reactions to this pandemic. Just one example is our realization that online “learning” may not be such a great vehicle for learning after all! I could go on and on. But thank you for that. It was very good.
Heidegger is an existence proof that a really awful human being can have some good ideas. Too bad that in addition to being a terrible person, he was also one of the worst writers ever. LOL.
I think there’s a problem with what is commonly referred to as “rational,” that being thinking that “the rational” must have to do only with externally observable facts. But our subjective experiences–that I prefer the blue screensaver to the yellow one, that I don’t like being poked in the ribs–these are as much facts about the world as are the speed of light in a vacuum or the population of Mumbai; it’s just that these are subjective facts. And it goes even further than that. States of being and experiencing can’t fully be captured by any enumeration of facts, objective or subjective. That’s why the arts are a necessary, essential form of communication. Wallace Stevens wrote that a poem should just escape the understanding, and what he meant, I think, is that it should evoke experience with the fullness that life experiences have. It always shocks me that people think that online learning can have the fullness, the significance, the meaningfulness of in-person learning with a teacher. Yeah, you can watch the video about baking a loaf of sourdough bread, but you can’t feel how the dough is supposed to feel when it is fully proofed but not overproofed. A teacher can show you that, let you experience it. There’s no substituting for that.
In the case of Hanushek and mainstream economics, it’s not even a rational way of thinking and certainly not scientific in the least.
Hanushek (and most economists) would not know science if it hit them in the head with an 🍎
I think it is not do much science perse but the tendancy to reduce everything to numbers.
Economists want to reduce everything to a single number (GDP)!
That’s just absurd.
This behavior seems to be common. Currently we have COVID-19 statistics for cases and deaths being compared side by side as if one got the disease and died within 24 hours. Has anyone heard of the average time between contracting the disease and deaths? I sure haven’t. In general the expectations are obvious when I see comments that say “cases have spiked but deaths have not.” They don’t bother to followup one or two weeks down the road to see if deaths spiked then.
great point, Steve
Many of the effects of covid19 may be chronic.
If someone gets lung damage and dies from complications 5 years later, will anyone attribute it to covid19?
Doubtful.
There is, however, a relationship between the creators of these tests and the money they reap.
YUP. The whole thing was a Gates business plan
I read a speech by him some 20 or 25 years ago in which he said that the costs of education were all in facilities and teacher salaries and that these could be reduced enormously by switching to computerized learning. Such synchronicity that this would just happen to make him a lot richer. LOL
Gee, and the costs of manufacturing, too, were once all tied up in antiquated facilities and salaries. Life is so much better now…
Brilliantly observed, Ginny!
Frederick Hess of AEI likely disagrees with the study.
AEI’s headquarters and the locations where its staff live –
a site map furnished by the Lincoln Project and provided to the activists among the 99%, odds?
It’s obvious that high PISA scores of 15-year-old determines nothing about overall GDP whatsoever. You can also see Japan as another example. A temporary spike in 2009 snd 2012 PISA scores didn’t have any influence on their GDP whatsoever. It was offset by aftershock of Wall Street Meltdown(collapse of Lehman Brothers) and Triple Disasters(9.0 Magnitude Earthquake, Nuclear Meltdown, and Tsunami). It’s a pity that those who were students at that time are now paid far less than their bosses and parents.
Ken, exactly right. Any thoughtful person would recognize that the test scores of 15 year old youth predict nothing as compared to macro events.
Gross Detestic Product
Mowing lawn and making bed
Will boost the GDP
But taking test, it should be said
Will only boost the fee
Always it’s about diverting dollars to the pockets of grifters while getting nothing of value in return.
Always
Always is a word
That’s always quite absurd
Except with grifter troll
We always pay the toll