Tom Loveless of the Brookings Institution calls on the OECD and PISA to stop permitting China to present data that does not represent the full population of students.

For one thing, only Shanghai is tested–and Shanghai is not representative of China.

Loveless writes that Shanghai’s #1 ranking on all subjects is misleading because it excludes the children of migrant workers.

He writes:

Shanghai has a school system that excludes most migrant students, the children of families that have moved to the city from rural areas of China.  And now for three years running, the OECD and PISA continue to promote a distorted picture of Shanghai’s school system by remaining silent on the plight of Chinese migrant children and what is one of the greatest human rights calamities of our time.

The numbers are staggering.  There are an estimated 230 million migrants in China.[1]  Approximately 5-6 million people have moved from rural areas to Shanghai since 2000.  Imagine a population the size of Los Angeles and Houston combined relocating to a city that was already larger than New York City—and in only thirteen years!  Shanghai’s population today is estimated at about 24 million people, with 13 million native residents and 11 million migrants.  For the most part, the migrants are poor laborers who fill the factories driving China’s export-driven economic boom.

The exclusionary school enrollment practices are rooted in China’s hukou (pronounced “who-cow”) system.  Although hukou dates back centuries, the current system was created by Mao Zedong’s regime in 1958 to control internal mobility in China.  Every family in China was issued a rural hukou by its home village or urban hukou by its home city, a document best understood as part domestic passport and part municipal license. 

The hukou controls access to municipal services.  Migrants in China with rural hukous are barred from a host city’s services, in particular, social welfare programs, healthcare providers, and much of the school system.  Hukous are transferred from generation to generation.  The children of migrants, even if born in Shanghai, receive their parents’ rural hukou, which their children, too, will someday inherit no matter where they are born.  As Kam Wing Chan, a Chinese migration and hukou expert at the University of Washington, puts it, “Under this system, some 700-800 million people are in effect treated as second class citizens, deprived of the opportunity to settle legally in cities and of access to most of the basic welfare and state-provided services enjoyed by regular urban residents.”

In addition, he says:

The barriers to migrants attending Shanghai’s high schools remain almost insurmountable.   High schools in Shanghai charge fees. Sometimes the fees are legal, but often in China, they are no more than bribes, as the Washington Post has reported.  Students must take the national exam for college (gaokao) in the province that issued their hukou.  An annual mass exodus of adolescents from city to countryside takes place, back to impoverished rural schools.  At least there, migrant kids might have a shot at college admission.  This phenomenon is unheard of anywhere else in the world; it’s as if a sorcerer snaps his fingers, and millions of urban teens suddenly disappear.

The toll on children and parents is staggering.  Families are torn apart.  Some migrant parents leave their children with relatives in villages when they initially move to cities in search of work.  The All China Women’s Federation estimates 61 million children are “left behinds,” as they are known in the country.  These children’s lives are marked by loneliness and despair.  A recent book, Diaries of China’s Left Behind Children, poignantly describes their plight.  The book caused a huge sensation in China.

What’s disgraceful is that OECD and PISA are complicit in allowing Shanghai to exclude a large part of its high school age students from the sample:

In 2010, Andreas Schleicher of the OECD revealed that the 2009 PISA was conducted in 12 provinces in China.  The data from mainland provinces other than Shanghai have never been released, and OECD’s list of participants in the 2009 PISA continues to omit them.  A Chinese website leaked purported scores from other provinces, but the scores have never been confirmed by PISA officials in Paris.

This shroud of secrecy is peculiar in international assessment.  Now the world has new data from the 2012 PISA.  The OECD has not disclosed if other Chinese provinces secretly took part in the 2012 assessment.  Nor have PISA officials disclosed who selected the provinces that participated.  Did the Chinese government pick the provinces?  Does the Chinese government decide which scores will be released?  In 2012, the BBC reported that theChinese government did not “allow” the OECD to publish PISA 2009 data on provinces other than Shanghai.  There is a lack of transparency surrounding PISA’s relationship with China.

Shanghai is portrayed as a paragon of equity in PISA publications.  A 2010 OECD publication,Strong Performers and Successful Reformers in Education, highlights model systems that the world should emulate.  Shanghai is singled out for praise.  One section on Shanghai is entitled, “Ahead of the pack in universal education.”  The city is described as an “education hub,” and the only discussion that even remotely touches upon migrants is the following:

“Graduates from Shanghai’s institutions are allowed to stay and work in Shanghai, regardless of their places of origin. For that reason, many ’education migrants now move to Shanghai mainly to educate their children.”[2]

That description is surreal.  PISA’s blindness to what is really going on in Shanghai was also evident in the official release of PISA’s latest scores.  The 2012 data appear in volumes organized by themes.  Volume II is entitled, PISA 2012 Results: Excellence through Equity, Giving Every Student the Chance to Succeed.  Shanghai is named as one jurisdiction where schools “achieve high mathematics performance without introducing greater inequities in education outcomes (p. 28)” and one with “above average socio-economic diversity (p. 30).”  In the 336 pages of this publication on equity, the word “migrant” appears only once, in a discussion of Mexico. The word “hukou” does not appear at all.

Is it possible that PISA officials are simply unaware of the hukou system and the media coverage cited above?  That’s doubtful, but even if it were the case, PISA’s own data shout out that something is wrong with Shanghai’s enrollment numbers.  PISA reports that 90,796 of Shanghai’s 15 year-olds are enrolled in school in grade 7 or above, out of a total population of 108,056 15 year-olds, producing an enrollment rate of about 84%. That’s comparable to other PISA participants.[3] Shanghai appears as inclusive as any other PISA participant.

What’s going on?

The only reasonable conclusion is this: officials in Shanghai are only counting children with Shanghai hukous as its population of 15 year-olds, about 108,000.  And the OECD is accepting those numbers.  It is as if the other children, numbering 120,000 or more, do not exist.  This is not a sampling problem.  PISA can sample all it wants from the official population.  Migrant children have been filtered out.  Professor Chan of Washington agrees with this hypothesis, saying in an email to me: “By the time PISA is given at age 15, almost all migrant children have been purged from the public schools.  The data are clear.”