Tom Loveless has been writing about international assessments for many years. He was quick to blow the whistle on China when the previous international test scores came out, noting that unlike the U.S. and most other nations, China was not testing a cross-section of its students.
In this article on Valerie Strauss’s Answer Sheet blog, Loveless calls out China again for rigging the outcomes to make its students #1.
China’s gains on the tests from 2015 to 2018 were so large as to be incredible, literally not credible.
So the typical change in a nation’s scores is about 10 points. The differences between the 2015 and 2018 Chinese participants are at least six times that amount. The differences are also at least seven times the standard deviation of all interval changes. Highly unusual…
The past PISA scores of Chinese provinces have been called into question (by me and others) because of the culling effect of hukou on the population of 15-year-olds — and for the OECD allowing China to approve which provinces can be tested. In 2009, PISA tests were administered in 12 Chinese provinces, including several rural areas, but only scores from Shanghai were released.
Three years later, the BBC reported, “The Chinese government has so far not allowed the OECD to publish the actual data.” To this day, the data have not been released. The OECD responded to past criticism by attacking critics and conducting data reviews behind closed doors. A cloud hangs over PISA scores from Chinese provinces. I urge the OECD to release, as soon as possible, the results of any quality checks of 2018 data that have been conducted, along with scores, disaggregated by province, from both the 2015 and 2018 participants.
The OECD allows China to hide data and game the system. This lack of transparency should not stand.
Diane Ravitch's blog 
I travel to China regularly. In the past China included only its top 2 cities, Shanghai, HK and Macau. Now its confidence extends to its top four provinces. There is no testing from vast rural undereducated PRC. This is like USA submitting only Massachusetts. Singapore’s underclass is overwhelmingly Malaysian. Their children are educated in Malaysia. Korea uses ‘hagwons’ cram tutorial schools open until 10 PM. Japan, however, seems fairly legitimate.
Apples and oranges in many cases.
LikeLike
Top 3 cities*
LikeLike
Doug Little : “Singapore’s underclass is overwhelmingly Malaysian. Their children are educated in Malaysia.”
Are you saying that Singapore sends its poorest Chinese students to Malaysia to be educated? Are they staying with relatives in Malaysia? Please elaborate.
I have never heard of this happening.
LikeLike
I am not sure I grasp that either. In her book ‘Empowered Teachers’ Darling-Hammond suggests that Singapore, Finland and Canada have worked on improving teaching, from the internship period and onward. It seems that very few changes have made to improve teaching conditions for our teachers.
There is room for improvement but teachers have to treated with much greater respect.
LikeLike
Everywhere on the planet, the poor score well below the rich or middle class on any test. Singapore has a substantial Chinese majority but much of the poorly paid work is done by Malaysians. They are like guest workers, they go home at night to Malaysia . What this means is when Singapore writes an international test, it is not held down by the scores of the poor children because most of the children of the poorest workers dont attend Singapore schools. The Singapore school children are a much different demographic than the other nations in PISA as a result.
Too hard to grasp?
LikeLiked by 1 person
so overwhelmingly easy to grasp, yet so transparently AVOIDED by those who would make money from testing: Everywhere on the planet, the poor score well below the rich or middle class…
LikeLike
One more try. The Chinese in Singapore stay put. In the AM workers from Malaysia cross bridges to work in Singapore, hotels, factory labour, cleaners etc, … same Malay workers go back at night. Their children are educated in Malaysia. THERFORE there are few poor children in Singapore schools. When you have a minimum of poor children, you will have much higher scores. This is DEMOGRAPHICS not curriculum or pedagogy.
LikeLike
In the US, the professional “reformers” talk about “zip codes,” as though zip codes are a causal factor in poor educational outcomes. Why not say “poverty,” and be honest?
LikeLike
I put in both of your quotes. One of my Malaysian friends, who lives in Kuala Lumpur, answered this way:
This is the first that I’ve heard of it!
I doubt it….
LikeLike
Our Japanese exchange student in 1994 was expecting to go home and spend three and four hours a day at the Juku preparing for college entrance exams. In those days we were being harangued for not being as good as the Japanese. Then it was Singapore. Then it was China. I need a break. Put cheese on my PISA.
LikeLike
I like my PISA with baloney, which is good because that’s the standard topping.
LikeLike
Magic mushrooms are another standard topping.
LikeLike
Oh, and buffalo turds.
LikeLike
Tell ya what SDP, how about I come, take ya out and pay for a baloney and mmshroom topping pizza. . . double shrooms of course! Hmmm I wonder if they’d to triple schrooms.
LikeLike
Japan tests their students in what is our 6th grade and tracks them into vocational or academic education. I would be surprised if PISA tests the vocational students.
LikeLike
The fact that the test scores can be games reflects more on the PISA organization and it’s test than it does on any country taking it.
This is Campbell’s law at work.
Why don’t people just admit the obvious rather than blaming individual countries for gaming the system?
PISA is a joke and so are the clowns who produce it.
LikeLike
I so agree.
LikeLike
I don’t understand why this is surprising since it’s exactly what virtually all the “high performing” charter school networks do when they claim superiority. They simply eliminate from their charters the students who do not test high. In fact, the system incentivizes that behavior and hugely rewards the charters who are the best at elimination and not teaching.
LikeLike
High stakes stdzd testing is an equal-opportunity corrupter. At tradl publics, we’ve had answer-changing cheating scandals, kids filling in random bubbles, & god knows what else passing under the radar.
LikeLike
Don’t give a rat’s ass what anyone’s scores, much less China’s, are on the completely invalid PISA test.
LikeLike
It’s an endless debate about a meaningless test.
Lke Trump’s tweets: full of sound and fury, but signifying nothing.
And yet, both Trump and the OECD economists have even the experts tied up in knots over their foolishness.
They have got everyone convinced that the world will end if they do not respond to every tweet and ever test result.
LikeLiked by 1 person
The biggest promoter of PISA is Andreas Schleicher, who works for the OECD and is in charge of PISA.
LikeLike
And A. Schleicher is the biggest shyster in regards to PISA.
LikeLike
PISA tests 15-year-olds.
There is no comparison between Chinese 15-year-olds vs the U.S. for the following reasons:
All children in China must attend school for at least nine years (not 13 as in the United States), known as the nine-year compulsory education, which is funded by the government. Compulsory education includes six years of primary education, starting at age six or seven, and three years of junior secondary education (junior middle school) for ages 12 to 15.
For 15-year-olds in China to be accepted to a high school after the end of their 9th grade mandatory education, they have to compete by taking a national test called the Zhongkao. This test is supposed to be voluntary and the volunteers often have parents that have supported and (often with harsh tiger/father-mother tactics) motivating them to do well in school.
If you want to get an idea of what Chinese tiger parents are like, I recommend reading the memoir “Battle Hymn of the Tiger Mother”, by Amy Chua. Amy (born in the U.S.) was raised by Chinese born tiger parents and she raised her children almost the same way but not exactly since she was married to an Jewish-American Caucasian that had his own opinions for raising their children but not too far off from Amy’s.
https://www.amychua.com/
There are only so many seats in China’s public non-mandatory high schools. For the government to pay for a high school student’s education after 9th grade, they have to score high on the Zhongkao. The students with the highest Zhongkao scores up to a cut off point are accepted. The rest are sent home or offered a vocational high school that will teach them a job skill that might lead to a job that does not need an academic high school education.
To get into a public academic high school in China requires that you end up with one of the top scores on the Zhonngkao.
The15-year-old, 10th-grade high school students in China tested by PISA had to be highly motived and earn high scores on the Zhongkao to make it to 10th grade.
At the end of 12th grade, to get into college, the best students from 8th grade that got into high school through the Zhongkao have to compete again with a test called the Gaokao that is considered one of the toughest high-stakes tests in the world. This also leads to high teen suicide rates because of the pressure.
“China’s education system is notorious for its focus on rote memorization and intense competition. Well-off families often enroll their kids in tutoring programs to try to give them an edge against their peers. Competition for getting into good schools, from elementary to college levels, is fierce.”
https://blogs.wsj.com/chinarealtime/2014/05/15/chinas-cutthroat-school-system-leads-to-teen-suicides/
The winners go to college. The losers go home (if they do not take their lives) and hopefully find a job even if it means going back in the fields to be a farmer.
Here’s the catch. China has been doing something like this since the Han Dynasty before Jesus Christ was born. There is no way the United States is going to copy what China does because U.S. cultural history isn’t the same as China’s. Education is not as important to many American parents as it is to Chinese parents.
LikeLike
“Catching up” doesn’t even apply, as the system you’re describing doesn’t test, & may not even educate many of its 15-y.o.’s. OECD should be showing a big old asterisk after China’s ranking.
LikeLike
We should show the OECD a big ol’ assterisk 🌕
LikeLike
By allowing China to hide students from the PISA, the OECD appears to want authoritarian China to look ominously superior. Heck, the whole point of the PISA pretty much has to be to feed a data arms race rivaling the nuclear race and the space race. How convenient for the “Race to the Top”, Nation at Risk crowd, pushing their privatizing, data-driven wage and labor rights suppression campaign against public education. Race to the race race! I’m sorry, I’ll stop there. I’m sure the billionaires buying public opinion with domestic thinky tanks and universities have nothing at all to do with influencing any international organizations into fear mongering that leads to a race for test scores against more repressive education systems. Uh huh. Sure. That couldn’t possibly happen. It’s just a coincidence. It’s not like they could just throw out China’s scores for cheating, right? Tossing scores is hard.
LikeLike
OECD is concerned about one thing and one thing only: keeping the money rolling in.
They are very good at playing the fear game to ensure that that is the case.
They have got all the politicians worldwide convinced that their test is some sort of benchmark for education and that the scores are a source of either pride or shame.
It is a deeply dishonest organization run by shysters.
LikeLike
But it’s run by economists, so what would you expect?
LikeLike
How many economists does it take to screw in a lightbulb?
None. Economists don’t actually do anything.
LikeLike
True, but how many economists does it take to screw up a lightbulb?
Just one.
LikeLike
Really, just one. Na, they wait for the night shift janitor to do it.
LikeLike
It does seem that China is gaming the PISA. On the other hand the U.S. has done poorly for quite some time. When the Los Angeles teachers had their strike one teacher said their library facilities were not adequate. It is affecting our students adversely. The affluent ones are okay but in poorer districts they are adversely affected by it.
LikeLike
The U.S. has NEVER been a top-ranking nation on international tests. I wrote a chapter about this in my 2010 book “The Death and Life of the Great American School System: How Testing and Choice Are Undermining Education.”
In the first international test in 1964, the subject was math only. A dozen nations took the test. The U.S. students came in last. As I wrote in that book, quoting Keith Baker of the U.S. Department of Education, fifty years later the U.S. had surpassed the other 11 nations with higher math scores on every economic metric. Go figure.
LikeLike
In my humble opinion , going back to the Coleman report, high and low achievement is far more a factor of life outside schools than inside schools. It is the demographics of rich and poor.
In Canada our policy is to spend more on poor schools than on middle class or affluent schools. It makes a difference but not as much as you would hope. Scores are still congruent to demographics.
Te way to make poor kids do better in school is mainly to eliminate poverty.
LikeLike
Yes, Doug! Yes, yes, yes. A million times yes!
LikeLike
But, but we’d have to ask the billionaires to give back a little of all they have stolen from others.
LikeLike
The comment thread at the Answer Sheet was dispiriting. Many Chinese jumping in to accuse American press of anti-Chinese bias & assuring us of their kids’ actual superiority in study habits/ intelligence despite apples/ oranges comparison—and many more Americans piggybacking on, ignoring the point of article & using it as soapbox to browbeat US ed system & bemoan inferior American ed ambition.
LikeLike
There are people who say one nation is exceptional. There are people who say another nation is exceptional. They all are all wrong.
LikeLike
Exceptionally wrong.
LikeLike
“Exceptionally Wrong”
You must be talking about Trump and “his” Republican Party regurgitated into “his” orange vomitus image built on a foundation of endless lies and rabid greed.
LikeLike
“Many Chinese jumping in to accuse American press of anti-Chinese bias & assuring us of their kids’ actual superiority in study habits/ intelligence despite apples/ oranges comparison—”
The Germans were quite prescient.
Their word for “orange” is Apfelsine, literally Apple from China”
LikeLike
Reblogged this on David R. Taylor-Thoughts on Education.
LikeLike
If it’s any consolation, US Asians had the 3rd highest average PISA score in the world (549) – you know, the ones Harvard said didn’t quite measure up on the “personality” scale.
LikeLike
Doug Little’s appears to be very confused when he makes the erroneous claim that, “Singapore’s underclass is overwhelmingly Malaysian. Their children are educated in Malaysia”. First, a clear distinction needs to be made between the term ‘Malaysian’ (a native or inhabitant of Malaysia) and the term ‘Malay’ (the ethnic group of Malays). ‘Malaysians’ certainly are not Singaporeans. Rather, Malaysians are citizens of Malaysia (a sovereign, multi-ethnic and multi-cultural country) and may be ethnically Malays or Bumiputeras (69%), Chinese (23%) or Indians (7%). On the other hand, Singaporeans are citizens of Singapore (another sovereign, multi-ethnic and multi-cultural country) and may be ethnically Chinese (76%), Malays (15%), or Indians (7%). In Singapore, under the Compulsory Education Act, a child who is a Singapore Citizen and of compulsory school age (above 6 years old and under 15 years old) must attend a national school regularly, i.e. attend a school in Singapore. I can assure Doug that those students in Singapore, who identify as Malay, attend school and study in Singapore, not Malaysia, and are included in PISA testing. Doug might further note that Malaysia is not a ‘poor’ country. Malaysia’s economy is one of the most competitive in Asia, ranking 6th in Asia and 20th in the world, higher than countries like Australia.
LikeLike
neville ellis: Thank you for your enlightening comment!!!!
Malaysia has many people who are considered middle class. However, the Bumiputra’s, get special treatments. There have been Chinese uprisings in the past and the Indian population is mostly having a rough time.
People who live in very rural areas are having difficult times since the government mostly recognizes those who live in cities.
Bumiputera or Bumiputra is a controversial Malaysian term to describe Malays.
Despite all of this Malaysia is doing pretty well economically. The Ringget has devalued against the US dollar. It was stable at 3.8 RM for one US dollar for 8 of the 9 years that I lived in KL.
LikeLike