Chris Tienkin, a professor at Seton Hall University, analyzed the data from the PISA international tests and concludes that they say more about American society than about American schools.
“Reform” policy makers like Arne Duncan and rightwing pundits like Michelle Rhee have used international scores to criticize and demean public schools and teachers. This tactic began with “A Nation at Risk,” which used the scores to predict the imminent decline of the American economy. It didn’t happen, of course, but the naysayers never stopped blaming the schools for their threat to our future, even as our economy boomed.
The biggest problem for our society is poverty, which affects test scores. But the test scores are the least of what matters. Inequality and poverty threaten our future and blight the lives of millions of Americans. The lucky few live in splendor; the desperate poor live in squalor. Public schools are not responsible for the disparity. At this point in history, the blame lies with the politics of greed.
Diane Ravitch's blog 
Many years ago — I can remember the day and where I was — I came to the realization that a democratic nation was probably doomed to elect its most average wits to every office … and so its only hope for survival was to raise the level of wisdom across the board. Then and there I dedicated myself to the task of education — in my own particular way …
But the policy of putting our dumbest and dastardest in charge of education has reached a point of no return and I fear its all over but the sinking of the ship of state.
But there must be a way to save some souls … and I’m sorry that classroom teachers find themselves manning the lifeboats, the last bastions of what’s left of democracy, but that is what has come to pass …
LikeLike
Although I am not skilled in statistics, I found Tienken’s analysis of the PISA compelling. My big take away is that it is erroneous to draw conclusions from aggregate data. Our PISA scores like so many other standardized tests reflect our socio-economic levels of our population. Our economic problems stem from our free trade policy with China with whom we cannot compete on a free market basis. We should not continue to give technology way to the Chinese. Our economic struggle does not reflect a failure of education policy, but a failure of our industrial policy. He goes on to that the US leads the world in patents, Nobel Prize science winners and ranks third on the entrepreneurial index.
Unless we change our economic policies, we will continue to have stagnant wages. Few of our young people are able to land secure jobs with benefits and pensions. In fact, many of them including STEM graduates remain unemployed. It is safe to say that it is unlikely that those that cannot get started will finish big. This has dangerous consequences for our future. Our standard of living will continue to slide downward unless our policymakers see the light.
LikeLike
1) Which is the most erroneous assumption in the test-based reform experiment?
a) cut scores are not arbitrary
b) the tests produce reliable and valid scores
c) students exert maximum effort
d) norm referencing trumps criterion referencing
LikeLike
“Analyzed the data from the PISA international tests and concludes that they say more about American society than about American schools.”
Those tests “say” nothing! The results “say” nothing! The data “say” nothing!
The tests/results/data have no meaning other than attempts by people to say that there is meaning when no valid meaning is available. Humans, actual people say many things about the testing event and the inevitable data that comes from the student interacting with a test. The only logical thing to say about that interaction is that any results gleaned from the results/data are, as the sage Noel Wilson puts it, “vain and illusory”. In other words all talk of, conclusions drawn from these epistemological and ontological frauds are COMPLETELY INVALID. Wilson proved so in his never refuted nor rebutted 1997 dissertation “Educational Standards and the Problem of Error” found at: http://epaa.asu.edu/ojs/article/view/577/700
Brief outline of Wilson’s “Educational Standards and the Problem of Error” and some comments of mine.
1. A description of a quality can only be partially quantified. Quantity is almost always a very small aspect of quality. It is illogical to judge/assess a whole category only by a part of the whole. The assessment is, by definition, lacking in the sense that “assessments are always of multidimensional qualities. To quantify them as unidimensional quantities (numbers or grades) is to perpetuate a fundamental logical error” (per Wilson). The teaching and learning process falls in the logical realm of aesthetics/qualities of human interactions. In attempting to quantify educational standards and standardized testing the descriptive information about said interactions is inadequate, insufficient and inferior to the point of invalidity and unacceptability.
2. A major epistemological mistake is that we attach, with great importance, the “score” of the student, not only onto the student but also, by extension, the teacher, school and district. Any description of a testing event is only a description of an interaction, that of the student and the testing device at a given time and place. The only correct logical thing that we can attempt to do is to describe that interaction (how accurately or not is a whole other story). That description cannot, by logical thought, be “assigned/attached” to the student as it cannot be a description of the student but the interaction. And this error is probably one of the most egregious “errors” that occur with standardized testing (and even the “grading” of students by a teacher).
3. Wilson identifies four “frames of reference” each with distinct assumptions (epistemological basis) about the assessment process from which the “assessor” views the interactions of the teaching and learning process: the Judge (think college professor who “knows” the students capabilities and grades them accordingly), the General Frame-think standardized testing that claims to have a “scientific” basis, the Specific Frame-think of learning by objective like computer based learning, getting a correct answer before moving on to the next screen, and the Responsive Frame-think of an apprenticeship in a trade or a medical residency program where the learner interacts with the “teacher” with constant feedback. Each category has its own sources of error and more error in the process is caused when the assessor confuses and conflates the categories.
4. Wilson elucidates the notion of “error”: “Error is predicated on a notion of perfection; to allocate error is to imply what is without error; to know error it is necessary to determine what is true. And what is true is determined by what we define as true, theoretically by the assumptions of our epistemology, practically by the events and non-events, the discourses and silences, the world of surfaces and their interactions and interpretations; in short, the practices that permeate the field. . . Error is the uncertainty dimension of the statement; error is the band within which chaos reigns, in which anything can happen. Error comprises all of those eventful circumstances which make the assessment statement less than perfectly precise, the measure less than perfectly accurate, the rank order less than perfectly stable, the standard and its measurement less than absolute, and the communication of its truth less than impeccable.”
In other words all the logical errors involved in the process render any conclusions invalid.
5. The test makers/psychometricians, through all sorts of mathematical machinations attempt to “prove” that these tests (based on standards) are valid-errorless or supposedly at least with minimal error [they aren’t]. Wilson turns the concept of validity on its head and focuses on just how invalid the machinations and the test and results are. He is an advocate for the test taker not the test maker. In doing so he identifies thirteen sources of “error”, any one of which renders the test making, giving and/or disseminating of results invalid. And a basic logical premise is that once something is shown to be invalid it is just that, invalid, and no amount of “fudging” by the psychometricians/test makers can alleviate that invalidity.
6. Having shown the invalidity, and therefore the unreliability, of the whole process Wilson concludes, rightly so, that any result/information gleaned from the process is “vain and illusory”. In other words start with an invalidity, end with an invalidity (except by sheer chance every once in a while, like a blind and anosmic squirrel who finds the occasional acorn, a result may be “true”) or to put in more mundane terms crap in-crap out.
7. And so what does this all mean? I’ll let Wilson have the second to last word: “So what does a test measure in our world? It measures what the person with the power to pay for the test says it measures. And the person who sets the test will name the test what the person who pays for the test wants the test to be named.”
In other words it attempts to measure “’something’ and we can specify some of the ‘errors’ in that ‘something’ but still don’t know [precisely] what the ‘something’ is.” The whole process harms many students as the social rewards for some are not available to others who “don’t make the grade (sic)” Should American public education have the function of sorting and separating students so that some may receive greater benefits than others, especially considering that the sorting and separating devices, educational standards and standardized testing, are so flawed not only in concept but in execution?
My answer is NO!!!!!
One final note with Wilson channeling Foucault and his concept of subjectivization:
“So the mark [grade/test score] becomes part of the story about yourself and with sufficient repetitions becomes true: true because those who know, those in authority, say it is true; true because the society in which you live legitimates this authority; true because your cultural habitus makes it difficult for you to perceive, conceive and integrate those aspects of your experience that contradict the story; true because in acting out your story, which now includes the mark and its meaning, the social truth that created it is confirmed; true because if your mark is high you are consistently rewarded, so that your voice becomes a voice of authority in the power-knowledge discourses that reproduce the structure that helped to produce you; true because if your mark is low your voice becomes muted and confirms your lower position in the social hierarchy; true finally because that success or failure confirms that mark that implicitly predicted the now self-evident consequences. And so the circle is complete.”
In other words students “internalize” what those “marks” (grades/test scores) mean, and since the vast majority of the students have not developed the mental skills to counteract what the “authorities” say, they accept as “natural and normal” that “story/description” of them. Although paradoxical in a sense, the “I’m an “A” student” is almost as harmful as “I’m an ‘F’ student” in hindering students becoming independent, critical and free thinkers. And having independent, critical and free thinkers is a threat to the current socio-economic structure of society.
LikeLike
I agree with you that “A Nation at Risk” began the heavy influence of the federal government in public education. “A Nation at Risk” lead directly to the raise of accountability and standards in the US, and eventually to No Child Left Behind (NCLB) in 2001. Even though our schools might not be the solution per say in fixing any societal ills (economy, war time nationalism, etc.), it has always been the place where society has placed the blame and where we need to fix the country as a whole.
With this background of schools being a place to fix America, it is no surprise when we see our test scores not at the top of these international tests, everyone freaks out. What people do not realize is that in the United States, we are a country of great diversity, and these tests might have biases against certain ethnic groups in the United States. But our scores in elementary and middle school seem tend to be in the top 10 internationally across subjects, so the real question we should be asking is why do they decline once our students hit high school? I know Dr. Tienkin is saying that poverty is causing our scores not to be as high, but I am sure other developed countries have students who live in poverty as well. I am not saying that the home life doesn’t affect a student’s performance on a test, but I am saying it is not the biggest factor we should worry about. We maybe sure turn to making sure we have highly qualified teachers in every classroom using evidence based practices to teach their students while collecting data to make data based instructional decisions. These practices cannot not stop after elementary and middle when the majority of standardized testing stops, but need to continue into high school to help ensure that all students will succeed.
LikeLike
THANK CLINTON for NAFTA, and THREE STRIKES and YOU’re OUT.
No Hillary for. She’s a DFER.
Beware of OBAMA and TPP.
It’s The Hunger Games, folks, front center.
LikeLike
Heck thank the BILLARIES for Standards and High-Stakes testing. I said, “NO! Can’t make me write standards and test items. I know the agenda.”
So what we gonna go now? How will you vote?
LikeLike