Yong Zhao, born and educated in China, is one of our most perceptive scholars of schools and society. He holds a professorship at the University of Kansas.
In this article, he reports the results of the latest international test, TIMSS. Once again, the East Asian nations topped the charts. Aside from 8th grade math, which are up, U.S. scores are unchanged.
TIMSS (Trends in International Mathematics and Science Study) beat PISA by two weeks. It just released its 2015 results. Within hours of the release, Google News has already collected over 10,000 news stories reacting to the results from around the world, some sad, some happy, some envious, and some confused. The biggest news is, however, nothing new: Children in East Asian countries best at maths. They were the best 20 years ago when TIMSS was first introduced in 1995. They were the best in all subsequent cycles.
Singapore, Hong Kong SAR, Korea, Chinese Taipei, and Japan are the top performers. In 4th grade, the lowest East Asian country is 23 points above the next best country, Northern Ireland for 4th grade, the same gap as was in 2011, and in 8th grade, a whopping 48 points lead ahead of the next best country, Russia, a 17 point increase from 31 in 2011.
Yong Zhao analyzes the reasons for their high scores.
Diane Ravitch's blog 
I was one of the top math substitutes in the Newport-Mesa U.S.D. I also taught other subjects. I had worked at least 500 times at Newport Harbor High School. In March, I was stopped from teaching because a girl who I had known for 3 years went bonkers and called me a liar. I have no idea why she said this and when I tried to find out, I was suspended from working at that school because the AP Jack Cusick decided the girl would be more comfortable without me and he also said that the kids don’t work well for me. This was total BS!! Because I complained and wanted an explanation, I was eventually fired. This shows how much our district really cares about who substituted their math classes!! Gordon Easton
Wow!
A difference of 25 or so points between country A and country B!
But what does that mean? Does it mean that the quality of the two countries’ educational systems are any different? Or does this difference point to one or two or many of the other possible factors that may have been at work:
How long is the test’s scale? 1000 points? Then a difference of 25 points would just mean 2,5 percent of the scale. In terms of number of tasks solved, how many tasks do these students differ? In some studies 25 points are the equivalent to 0,5 tasks solved.
How many of the “poor” students found better solutions than the test-makers and got punished for this? (Many of the answers scored as “right” are ambivalent, or one of several right solutions, or just worng answers). How many students gave a heck for filling out another bunch of tests which do not even count for them personally but only for the Great Nation? How many students were asked by their schools to stay home in order to elevate the overall mean score in country A and B? How much did the tests differ in text-length for language reasons, and afforded different reading times? For how many students was the test language their second language? How did mathematical techniques of forcing the distribution of scores under the bell-curve affect the outcome?
I could extend this list of factors contributing to the “difference” of 25 test points between two countries.
Which of these factors can be hold accountable? For what reason?
Why does nobody look into the test’ validity anymore?
Why do we settle with the “truth” voiced by the PR departments of the test-sellers?
“But what does that mean?”
Thanks for the opening George! Others may rue that opening but here is my response to your question: Onto-epistemologically speaking ABSOLUTELY NOTHING.
Noel Wilson, in his 1997 dissertation has shown that the standards and standardized testing regime is so fraught with errors, falsehoods and psychometric fudging that any results and conclusions drawn from those results are COMPLETELY INVALID!
If you have not read Wilson’s never refuted nor rebutted (and I’ve been searching and asking for over 15 years for any rebuttal or refutation and not found any whatsoever) treatise:
“Educational Standards and the Problem of Error” found at: http://epaa.asu.edu/ojs/article/view/577/700
Brief outline of Wilson’s “Educational Standards and the Problem of Error” and some comments of mine. (updated 6/24/13 per Wilson email)
A description of a quality can only be partially quantified. Quantity is almost always a very small aspect of quality. It is illogical to judge/assess a whole category only by a part of the whole. The assessment is, by definition, lacking in the sense that “assessments are always of multidimensional qualities. To quantify them as unidimensional quantities (numbers or grades) is to perpetuate a fundamental logical error” (per Wilson). The teaching and learning process falls in the logical realm of aesthetics/qualities of human interactions. In attempting to quantify educational standards and standardized testing the descriptive information about said interactions is inadequate, insufficient and inferior to the point of invalidity and unacceptability.
A major epistemological mistake is that we attach, with great importance, the “score” of the student, not only onto the student but also, by extension, the teacher, school and district. Any description of a testing event is only a description of an interaction, that of the student and the testing device at a given time and place. The only correct logical thing that we can attempt to do is to describe that interaction (how accurately or not is a whole other story). That description cannot, by logical thought, be “assigned/attached” to the student as it cannot be a description of the student but the interaction. And this error is probably one of the most egregious “errors” that occur with standardized testing (and even the “grading” of students by a teacher).
Wilson identifies four “frames of reference” each with distinct assumptions (epistemological basis) about the assessment process from which the “assessor” views the interactions of the teaching and learning process: the Judge (think college professor who “knows” the students capabilities and grades them accordingly), the General Frame-think standardized testing that claims to have a “scientific” basis, the Specific Frame-think of learning by objective like computer based learning, getting a correct answer before moving on to the next screen, and the Responsive Frame-think of an apprenticeship in a trade or a medical residency program where the learner interacts with the “teacher” with constant feedback. Each category has its own sources of error and more error in the process is caused when the assessor confuses and conflates the categories.
Wilson elucidates the notion of “error”: “Error is predicated on a notion of perfection; to allocate error is to imply what is without error; to know error it is necessary to determine what is true. And what is true is determined by what we define as true, theoretically by the assumptions of our epistemology, practically by the events and non-events, the discourses and silences, the world of surfaces and their interactions and interpretations; in short, the practices that permeate the field. . . Error is the uncertainty dimension of the statement; error is the band within which chaos reigns, in which anything can happen. Error comprises all of those eventful circumstances which make the assessment statement less than perfectly precise, the measure less than perfectly accurate, the rank order less than perfectly stable, the standard and its measurement less than absolute, and the communication of its truth less than impeccable.”
In other words all the logical errors involved in the process render any conclusions invalid.
The test makers/psychometricians, through all sorts of mathematical machinations attempt to “prove” that these tests (based on standards) are valid-errorless or supposedly at least with minimal error [they aren’t]. Wilson turns the concept of validity on its head and focuses on just how invalid the machinations and the test and results are. He is an advocate for the test taker not the test maker. In doing so he identifies thirteen sources of “error”, any one of which renders the test making/giving/disseminating of results invalid. And a basic logical premise is that once something is shown to be invalid it is just that, invalid, and no amount of “fudging” by the psychometricians/test makers can alleviate that invalidity.
Having shown the invalidity, and therefore the unreliability, of the whole process Wilson concludes, rightly so, that any result/information gleaned from the process is “vain and illusory”. In other words start with an invalidity, end with an invalidity (except by sheer chance every once in a while, like a blind and anosmic squirrel who finds the occasional acorn, a result may be “true”) or to put in more mundane terms crap in-crap out.
And so what does this all mean? I’ll let Wilson have the second to last word: “So what does a test measure in our world? It measures what the person with the power to pay for the test says it measures. And the person who sets the test will name the test what the person who pays for the test wants the test to be named.”
In other words it attempts to measure “’something’ and we can specify some of the ‘errors’ in that ‘something’ but still don’t know [precisely] what the ‘something’ is.” The whole process harms many students as the social rewards for some are not available to others who “don’t make the grade (sic)” Should American public education have the function of sorting and separating students so that some may receive greater benefits than others, especially considering that the sorting and separating devices, educational standards and standardized testing, are so flawed not only in concept but in execution?
My answer is NO!!!!!
One final note with Wilson channeling Foucault and his concept of subjectivization:
“So the mark [grade/test score] becomes part of the story about yourself and with sufficient repetitions becomes true: true because those who know, those in authority, say it is true; true because the society in which you live legitimates this authority; true because your cultural habitus makes it difficult for you to perceive, conceive and integrate those aspects of your experience that contradict the story; true because in acting out your story, which now includes the mark and its meaning, the social truth that created it is confirmed; true because if your mark is high you are consistently rewarded, so that your voice becomes a voice of authority in the power-knowledge discourses that reproduce the structure that helped to produce you; true because if your mark is low your voice becomes muted and confirms your lower position in the social hierarchy; true finally because that success or failure confirms that mark that implicitly predicted the now self-evident consequences. And so the circle is complete.”
In other words students “internalize” what those “marks” (grades/test scores) mean, and since the vast majority of the students have not developed the mental skills to counteract what the “authorities” say, they accept as “natural and normal” that “story/description” of them. Although paradoxical in a sense, the “I’m an “A” student” is almost as harmful as “I’m an ‘F’ student” in hindering students becoming independent, critical and free thinkers. And having independent, critical and free thinkers is a threat to the current socio-economic structure of society.
It is an important exercise with points the press is unlikely to care about. It looks beyond the scores for reasons. Needless to say, his conclusion is perfect. I hope he is invited to offer your explanation to the philanthropic and tech communities. In the meantime, get those chopsticks into the hands of kids, the sooner the better.
The only real measure of the educational success of a country is in the performance of its adults. In science, technology, mathematics, engineering, literature, medicine, entertainment, sports etc., the United States does very well indeed and is always at the top or near the top. We have nothing to be ashamed of. With the current emphasis on test scores, we might soon be more like the countries with the high test scores and less like ourselves.
I know of NO written test which can evaluate the worth to society of ANY person, child or adult.
People have forgotten what real education is about.
When a Nation at Risk came out ALL schools, ALL teachers were failures.
When we put a man of t he moon, not one word of the expertise of our scientists who helped put them there who wer eeducated in American schools..
We were to emulate the great Russian schools. Where would we be now had we done that.
When the Japanese made better autos than America did at that time, An American went there to show them how to do it, we were to emulate the great Japanese schools. Now that the Japanese economy has stagnated, not one word about the great Japanese schools now.
Our schools are, have been and it seems that it will continue to be the scapegoat for politicians who are inept,ignorant, blind, myopic ad nauseum.
They will not be satisfied, Trump especially until they destroy our planet and everything else which makes life worth living.
TRUTH; irrelevant. How much lower can we get than that?
Terrifying in its implications.
American education has been the eternal scapegoat while the accomplishments of our citizenry are taken for granted.
Well said!
“I know of NO written test which can evaluate the worth to society of ANY person, child or adult.”
Bingo, bangle, boingo! We have a winner! Give that man a Kewpie Doll!
A smallish group of mono ethnic countries with declining populations , and in the case of Japan, decades of stagnation.
Maybe math doesn’t matter that much.
Just proves that test scores don’t cause a growing economy
I don’t know how countries are compared at TIMSS. But in my (college) teaching experience in the US, there is a noticeable difference between Asian students and others. The difference is not just in showing greater respect to the teacher, or working harder than others, but in their appreciation of the subject.
Here’s my concrete experience this semester in my “Introduction to higher mathematics” class.
One student worked consistently throughout the whole semester, a Korean student. She is a mother of two. One day, she had to bring her middle school kids to class, since school was closed on that day. I had students bringing their kids to class before, but this time was different. Both kids paid very close attention during the whole class, and I thought, maybe they are drawing a picture of the class. But no, they actually took notes! Not only that, the mother told me that on their way home, the kids said, this was the coolest class, and at home they insisted in explaining to their dad what they had learned in “school”.
So I wouldn’t dismiss the TIMSS result by saying “it’s just based on test results, and we don’t care about that”, since it’s possible, there is an important student attitude behind the great test scores that may deserve our attention.
Similarly, it’s not exactly wise to dismiss the great international ranking of Finnish students: that country really is doing something worthwhile in education.
If we want to look at the list of reasons in the article, I think 5) is significant
5 East Asian students do not necessarily receive more classroom instruction compared to the U.S., Australia, Canada or England. In 4th grade, for example, Korea spends the least amount of time at 100 hours, Chinese Taipei spends 128 hours, Japan 151 hours, Hong Kong 159, Singapore 201. The International Average is 157 hours. In comparison, the U.S. spends 216 hours, Australia 202 hours, Canada 196, and England 189 hours.
In many countries, kids are simply overworked in school, and the US is an extreme example.
One can summarize the article’s findings by saying that perhaps there’s less pressure on kids in East Asian countries to learn math.
This semester I have also been teaching math for elementary math teachers, and I had to use workbooks based on the “Singapore” model. These workbooks are more low-key than those my kids studied from. There is less pressure, slower pace in these Singapore workbooks, hence more time can be spent on understanding the material.
But: the Singapore work books are still too much, imo. My suspicion is that perhaps the Korean books contain the right amount of material.
How much does the U.S. government spend on participation in flawed and useless international league tables?