James Harvey, director of the National Superintendents Roundtable, wrote a terrific article about international assessments in Valerie Strauss’s Answer Sheet.
“These assessments were never intended to line up and rank nations against each other like baseball standings.
“That’s right. The statisticians and psychometricians who dreamed up these assessments 50 years ago stated explicitly that the question of whether “the children of country X [are] better educated that those of country Y” was “a false question” due to the innumerable social, cultural, and economic differences among nations. But, hey, that’s just a detail.”
“2. The “international average” isn’t what you think it is. It’s not a weighted average of all the students in the world, but an average of the national averages.
“This means that when calculating the “international average,” the 5,600 students in Lichtenstein, the 700,000 in Ireland, the 860,000 in Finland, the 5 million in Canada, and the 14 million in Japan carry exactly the same weight as the 56 million students in the United States.”
And here is more:
“3. These assessments compare apples and oranges.
“Do you think there’s anything to be learned from comparing the average performance of 5,600 wealthy white students in Lichtenstein with 56 million diverse students in the United States? Really? How about comparing our students with students in corrupt dictatorships like Kazakhstan, religious monarchies like Qatar, or the wealthiest city in China (Shanghai) after it has driven the children of low-income migrants back to their home provinces? As a report released in January by the Horace Mann League and the National Superintendents Roundtable, “School Performance in Context: The Iceberg Effect,” makes clear, these are just a few of the peculiar comparisons that lie behind these international assessment results.”
“5. The horse-race tables ignore differences in poverty, inequity, and social stress among nations.
“Fifty years of research in the United States and abroad documents a powerful correlation between low student achievement and poverty and disadvantagement. Yet reports on these international assessments blandly turn a blind eye on the implications of this research. The data are clear: Poverty rates among American students are five times higher than they are in Finland. China aside, we have the highest rates of income inequality in the nine nations examined in The Iceberg Effect. The rate of violent deaths in American communities is eight times the average rate in the other eight nations and 13 times greater than it is in Japan. All of that is ignored in the orgy of publicity organized by the sponsoring agencies of these assessments to highlight their findings.”
All in all, a brilliant analysis of the limitations of these tests that have promoted the deeply flawed agenda of test and punish.