Arthur Camins is director of the Center for Innovation in Engineering and Science Education at the Stevens Institute of Technology in Hoboken, New Jersey.

He left the following comment about the PISA results:

The release of NAEP, TIMSS and PISA scores always produces debate. How do we compare with others (and on what)? Among us, who has improve and who has not? Are we improving and, if so, are we improving fast enough?

You and others have cogently argued that quickly leaping to favored policy implications usually lacks much evidence and is often misleading.

Arguments in a democracy are natural and could be healthy, but I worry we are not making much progress when it comes to current education policy. Maybe a dose of engineering design thinking can help.

An essential step in such thinking is defining and delimiting the problems. The biggest problem with education is the US is not test scores. Rather, three central problems plague public education the United States. The most dramatic is inequity. There are vast inequities in educational resources and in the conditions of students’ lives, resulting in persistent race- and class-based disparities in educational outcomes.

Second, we are far too focused on a narrow range of outcomes — reading and math test scores — and not enough on a broader range of subject matter or essential domains, such as critical thinking, creativity and collaborative skills. Third, we gravitate toward partial quick solutions, rather than thinking systemically and having the patience allow strategies time to develop, take hold and be refined.

Next, we need to consider both values and technical constraints for ideal solutions. For example, we need to ask what mix of collaborative and competitive strategies align with our values and research on systems that have been successful in sustaining significant educational improvement.
In addition, since ideal solutions always prove better in theory than in practice, we need to plan for optimization– repeated cycles of testing, redesign and retesting.

Finally, to make progress we need mobilize the necessary political will. To do so, we need to hear more about common sense, high-leverage solutions– framed as messages that respect people’s intelligence and tap into their values, aspirations and sense of fairness.

I made several suggestions about these messages last week on the Washington Post’s education blog, The Answer Sheet:http://www.washingtonpost.com/blogs/answer-sheet/wp/2013/11/30/how-thinking-like-an-engineer-can-help-school-reform/

The Policy Consortium in the UK  has a good overview of the British response to the PISA scores.

Each political party is pointing fingers at the other for the scores not being as high as they would wish.

The Conservatives say it is Labor’s fault.

The Labor party says it is the fault of the Conservatives.

But here are some good takeaways.

“… it is important to attract the most talented teachers to the most challenging classrooms and teacher shortage and disciplinary climate are inter-related.

Moreover, despite what Michael Gove asserts, a qualified teaching force is a key driver of quality and performance, the PISA data shows. Two findings are key here: first, the quality of a school (or college) cannot exceed the quality of its teachers. Second, principals of disadvantaged schools have difficulty attracting well qualified teachers, so students suffer doubly.

And more:

Much has been said in the media about the influx of migrant groups such as the Roma undermining provision for the indigenous population (all part of the attack on the EU by right-wing media). But, again, the PISA report shows this is first and foremost an issue of resources. “The concentration of immigrants in a school is not associated, in itself, with poor performance”, it says.

Nor is it a question of fairness. High-performing school systems tend to allocate resource more equitably across advantaged/ disadvantaged schools. Also, combining high performance with a high degree of equity is possible – it happens in some countries.

Another observation, with which all political parties would claim to be in tune, is that “schools with more autonomy over curricula and assessments tend to perform better when they are part of a school system with greater collaboration between principals and teachers”.

So, rather than sniping across the political garden fence, politicians should try to build a consensus around policy options that improve performance and equity. Four such actions which the PISA report clearly identifies are:

• targeting low performance regardless of socio economic status
• targeting disadvantaged pupils through additional resources or finance
• improving the quality of teaching staff, focusing on time for teachers themselves to learn
• including marginalised students in mainstream education

And here is another important finding:

From the outset, the need for a good start is clearly identified. For example, the report shows, one year of pre-school improves performance in maths by one year of schooling.

Better staff-student relations are associated with greater student engagement. “Too many students do not make the most of the learning opportunities available to them because they are not engaged with school and learning. Drive, motivation and confidence in oneself are essential if students are to fulfil their potential”.

The news reports say that the test scores of American students on the latest PISA test are “stagnant,” “lagging,” “flat,” etc.

The U.S. Department of Education would have us believe–yet again–that we are in an unprecedented crisis and that we must double down on the test-and-punish strategies of the past dozen years.

The myth persists that once our nation led the world on international tests, but we have fallen from that exalted position in recent years.

Wrong, wrong, wrong.

Here is the background history that you need to know to interpret the PISA score release, as well as Secretary Duncan’s calculated effort to whip up national hysteria about our standing in the international league tables.

The U.S. has NEVER been first in the world, nor even near the top, on international tests.

Over the past half century, our students have typically scored at or near the median, or even in the bottom quartile.

International testing began in the mid-1960s with a test of mathematics. The First International Mathematics Study tested 13-year-olds and high-school seniors in 12 nations. American 13-year-olds scored significantly lower than students in nine other countries and ahead of students in only one. On a test given only to students currently enrolled in a math class, the U.S. students scored last, behind those in the 11 other nations. On a test given to seniors not currently enrolled in a math class, the U.S. students again scored last.

The First International Science Study was given in the late 1960s and early 1970s to 10-year-olds, 14-year-olds, and seniors. The 10-year-olds did well, scoring behind only the Japanese; the 14-year-olds were about average. Among students in the senior year of high school, Americans scored last of eleven school systems.

In the Second International Mathematics Study (1981-82), students in 15 systems were tested. The students were 13-year-olds and seniors. The younger group of U.S. students placed at or near the median on most tests. The American seniors placed at or near the bottom on almost every test. The “average Japanese students achieved higher than the top 5% of the U.S. students in college preparatory mathematics” and “the algebra achievement of our most able students (the top 1%) was lower than that of the top 1% of any other country.” (The quote is from Curtis C. McKnight and others, The Underachieving Curriculum: Assessing U.S. Mathematics from an International Perspective, pp. 17, 26-27). I summarized the international assessments from the mid-1960s to the early 1990s in a book called National Standards in American Education: A Citizen’s Guide (Brookings, 1995).

The point worth noting here is that U.S. students have never been top performers on the international tests. We are doing about the same now on PISA as we have done for the past half century.

Does it matter?

In my recent book, Reign of Error, I quote extensively from a brilliant article by Keith Baker, called “Are International Tests Worth Anything?,” which was published by Phi Delta Kappan in October 2007. Baker, who worked for many years as a researcher at the U.S. Department of Education, had the ingenious idea to investigate what happened to the 12 nations that took the First International Mathematics test in 1964. He looked at the per capita gross domestic product of those nations and found that “the higher a nation’s test score 40 years ago, the worse its economic performance on this measure of national wealth–the opposite of what the Chicken Littles raising the alarm over the poor test scores of U.S. children claimed would happen.” He found no relationship between a nation’s economic productivity and its test scores. Nor did the test scores bear any relationship to quality of life or democratic institutions. And when it came to creativity, the U.S. “clobbered the world,” with more patents per million people than any other nation.

Baker wrote that a certain level of educational achievement may be “a platform for launching national success, but once that platform is reached, other factors become more important than further gains in test scores. Indeed, once the platform is reached, it may be bad policy to pursue further gains in test scores because focusing on the scores diverts attention, effort, and resources away from other factors that are more important determinants of national success.” What has mattered most for the economic, cultural, and technological success of the U.S., he says, is a certain “spirit,” which he defines as “ambition, inquisitiveness, independence, and perhaps most important, the absence of a fixation on testing and test scores.”

Baker’s conclusion was that “standings in the league tables of international tests are worthless.”

I agree with Baker. The more we focus on tests, the more we kill creativity, ingenuity, and the ability to think differently. Students who think differently get lower scores. The more we focus on tests, the more we reward conformity and compliance, getting the right answer.

Thirty years ago, a federal report called “A Nation at Risk” warned that we were in desperate trouble because of the poor academic performance of our students. The report was written by a distinguished commission, appointed by the Secretary of Education. The commission pointed to those dreadful international test scores and complained that “on 19 academic tests American students were never first or second and, in comparison with other industrialized nations, were last seven times.” With such terrible outcomes, the commission said, “the educational foundations of our society are presently being eroded by a rising tide of mediocrity that threatens our very future as a Nation and a people.” Yet we are still here, apparently the world’s most dominant economy. Go figure.

Despite having been proved wrong for the past half century, the Bad News Industry is in full cry, armed with the PISA scores, expressing alarm, fright, fear, and warnings of imminent economic decline and collapse.

Never do they explain how it was possible for the U.S. to score so poorly on international tests again and again over the past half century and yet still emerge as the world’s leading economy, with the world’s most vibrant culture, and a highly productive workforce.

From my vantage point as a historian, here is my takeaway from the PISA scores:

Lesson 1: If they mean anything at all, the PISA scores show the failure of the past dozen years of public policy in the United States. The billions invested in testing, test prep, and accountability have not raised test scores or our nation’s relative standing on the league tables. No Child Left Behind and Race to the Top are manifest failures at accomplishing their singular goal of higher test scores.

Lesson 2: The PISA scores burst the bubble of the alleged “Florida miracle” touted by Jeb Bush. Florida was one of three states–Massachusetts, Connecticut, and Florida–that participated in the PISA testing. Massachusetts did very well, typically scoring above the OECD average and the US average, as you might expect of the nation’s highest performing state on NAEP. Connecticut also did well. But Florida did not do well at all. It turns out that the highly touted  “Florida model” of testing, accountability, and choice was not competitive, if you are inclined to take the scores seriously. In math, Florida performed below the OECD average and below the U.S. average. In science, Florida performed below the OECD average and at the U.S. average. In reading, Massachusetts and Connecticut performed above both the OECD and U.S. average, but Florida performed at average for both.

Lesson 3: Improving the quality of life for the nearly one-quarter of students who live in poverty would improve their academic performance.

Lesson 4: We measure only what can be measured. We measure whether students can pick the right answer to a test question. But what we cannot measure matters more. The scores tell us nothing about students’ imagination, their drive, their ability to ask good questions, their insight, their inventiveness, their creativity. If we continue the policies of the Bush and Obama administrations in education, we will not only NOT get higher scores (the Asian nations are so much better at this than we are), but we will crush the very qualities that have given our nation its edge as a cultivator of new talent and new ideas for many years.

Let others have the higher test scores. I prefer to bet on the creative, can-do spirit of the American people, on its character, persistence, ambition, hard work, and big dreams, none of which are ever measured or can be measured by standardized tests like PISA.