Stephen Krashen is an expert on linguistics and literacy. He read Arthur Levine’s scathing criticism of US test scores on the PISA exam and decided to do his own analysis. He decided Levine was far too negative.

Krashen decided that: “the US does quite well, controlling for SES. And maybe American scores are ‘just right.’”

He writes:

“In a recent article in the Wall Street Journal, Arthur Levine discusses the performance of high socio-economic status (SES) students on the PISA math examination, thus controlling for the effect of poverty (the PISA is an exam given to 15 year olds in countries throughout the world). Levine concludes that high social class American students fall in “the middle of the pack” in PISA mathematics.

“Levine’s definition of high social class was having at least one parent with a college education. After reading Levine’s article, I decided to do my own analysis. I used a different measure of SES: the PISA index of economics, social and cultural status. I looked at reading scores for students in 66 countries who were at the 75th percentile of this measure, in other words the upper quarter of socio-economic status.

“According to my calculations, students in only 12 “countries and economies” scored significantly higher than American students on PISA reading and students in 44 “countries and economies” had significantly lower scores. Levine says the US scored in the “middle of pack” in math. Controlling for poverty, they certainly did not score in the middle of the pack in reading but were well within the upper quarter. (See note 1 below; I was unable to find the data necessary to do an analysis of mathematics scores controlling for SES in this way.)

“Yong Zhao of the University of Oregon has reported that countries that score high on international tests score low on measures of “perceived entrepreneurial capabilities.” This result is consistent with research cited by D. K. Simonton in his book Genius, Creativity and Leadership: an optimal amount of formal education is best for creative accomplishment in science and the arts and humanities – not too much and not too little. Simonton also concludes that that those who achieve high scholastic honors do not always attain eminence in their work.

“Maybe US scores are just right.”

Note 1: American students scored 569 on the PISA reading test, with a standard error of 4.6. As mentioned, 12 “countries and economies” scored significantly higher (i.e. their scores fell outside the 95% confidence interval around the US’ score, 560 to 578).

Of the 12 scoring higher than the US, several were not countries. Shanghai is a city, with 23 million people, about 1.5% of the population of China and is a clear outlier: Even Shanghai students in the lowest quartile in socio-economic status scored 500, close to the overall average for all OECD countries.

Singapore is considered a “city-state” and has a population of five million. Hong Kong is a “special administrative region (SAR)” of China with a population of seven million. Both Singapore and Hong Kong have fewer people than Los Angeles County, states of Michigan or Georgia, all around 10 million.

Thus, only nine actual countries did better than the US. Note also that other countries doing better than the US also have small populations: Finland, 5.5 million, New Zealand, 4.5 million, and Belgium, 11 million.

Sources:

Levine, A. 2012. The Suburban Education Gap. Wall Street Journal, November 14, 2012.

Simonton, D.K. 1984. Genius, Creativity and Leadership. Cambridge: Harvard University Press.

PISA 2009. Overcoming Social Background. Programme for International Student Assessment. http://www.oecd.org/pisa/pisaproducts/pisa2009/pisa2009resultsovercomingsocialbackgroundequityinlearningopportunitiesandoutcomesvolumeii.htm

Zhao, Y. 2012. Flunking innovation and creativity. Phi Delta Kappan 94 (1): 56-60.

The scores: (from: Table 11.1.1, PISA 2009, p. 152).

Shanghai
613 (2.8)
Finland
597 (2.2)
Singapore
597 (2.1)
New Zealand
595 (2.8)
Korea
595 (3.4)
Hong Kong
592 (2.5)
Japan
590 (3.0)
Canada
588 (1.7)
Australia
584 (2.7)
Belgium
583 (2.2)
Netherlands
575 (5.4)
France
572 (4.0)
Switzerland
569 (3.0)
USA
569 (4.6)
Norway
568 (2.9)
Germany
567 (2.8)
Iceland
567 (2.0)
Poland
565.(3.2)
Sweden
565 (3.2)
Ireland
562 (2.8)
UK
561 (3.2)
Liechtenstein
560 (4.5)
Estonia
559 (2.8)
Hungary
559 (3.6)
Italy
556 (1.7)
Taipei
555 (2.9)
Israel
554 (3.4)
Denmark
554 (2.8)
Portugal
551 (3.4)
Greece
550 (3.1)
Slovenia
550 (1.7)
Luxembourg
547 (1,7)
Austria
545 (3.3)
Czech Rep
545 (3.3)
Austria
545 (3.3)
Czech Rep
545 (3.3)
Slovak Rep
543 (2.7)
Spain
543.(2.0)
Latvia
541 (3.3)
Leichtenstein
541 (3.3)
Macao-China
540 (1.4)
Croatia
539 (3.1)
Dubai
536 (2.4)
Lithuania
530 (3.1)
Turkey
522 (4.5)
Russian Fed.
519 (3.2)
Bulgaria
512 (6.5)
Serbia
501 (2.5)
Trinidad/Tobango
496 (2.3)
Uruguay
495 (3.1)
Romania
488 (4.7)
Mexico
485 (1.9)
Brazil
474 (3.9)
Argentina
473 (7.1)
Columbia
473 (3.9)
Thailand
469 (2.6)
Indonesia
468 (3.5)
Tunisia
462 (3.4)
Albania
458 (4.8)
Kazakstan
452 (4.2)
Qatar
450 (1.4)
Indonesia
447 (4.6)
Peru
437 (5.2)
Panama
436 (7.7)
Azerbaijan
413 (4.0)
Kyrgystan
377 (4.2)