Politico Morning Education reports:
U.S. SCORES IN READING, MATHEMATICS AND SCIENCE LITERACY REMAINED ESSENTIALLY FLAT FROM 2015 in the latest Program for International Student Assessment results, but U.S. rankings improved because other education systems worsened.
— The 2018 PISA results showed U.S. average scores in reading and science literacy were higher than the average of about three dozen mostly industrialized countries making up the Organization for Economic Cooperation and Development, which develops and coordinates the assessment. But U.S. average math scores were lower than the OECD average.
— PISA, an international assessment administered every three years, measures 15-year-old students’ literacy in the three disciplines and is designed to provide a global view of U.S. students’ performance compared to their peers in nearly 80 education systems.
— “If I communicated nothing, I hope I communicated that we are struggling in math in comparison to our competitors around the world,” Peggy G. Carr, the associate commissioner of assessments for the National Center for Education Statistics, told reporters in a call before the results were released. Nicole Gaudiano has more.
Diane Ravitch's blog 
Our school has DOUBLED time devoted to math for the past ten years. All kids get TWO full periods of math per day. I wonder how many other American schools have done this too. If this is widespread, it makes the flat-lining of the scores even more scandalous.
in our school not only math but literacy were suddenly doubled: no one even arguing that the courses then cut from the curricula to make this happen might be just as important. Hasta la vista civics, government, social studies, business math, art, gym, languages…
The world of possibilities for this generation does not hinge on test scores in math, reading, and science literacy and the ranking of the performances of students in the US with “peers in 80 education systems.”
In my view, the scandal is not in “flatlined scores in spite of more time on math” but in thinking that more time SHOULD increase test scores.
There is also in all OECD comparisons a refusal to acknowledge that educational systems in 80 countries are not the same and that higher test scores may be the result of the many, many differences in those systems–the resources they have, and the populations they serve. Translations of test questions so they have comparable meanings, in addition to cultural referents, is a chronic problem in these tests. You can see some of the questions for yourself at the OECD website.
OECD reports are increasingly preoccupied with tying test scores to teaching methods, curriculum, and other minutia as if to push many “systems of education” in the same direction–one that raises test scores. OECD is at the center of GERM…the Global Education Reform Movement identified by Pasi Sahlberg and not in a favorable light.
When the most powerful nation in the world has average scores, something is wrong with the measure.
“. . . something is wrong with the measure.”
Because at the very fundamental conceptual basis, at the onto-epistemological footing there is no “measure”. Nothing is being measured.
The measurement meme, the emphasis on a supposed objective “measure” is totally false. But hey, the psychometricians* have been making good money by selling that snake oil nonsense for many years now.
*better written as “psychomeretricians”.
Duane, are you saying no test has merit?
Ponderosa,
YES!
Ponderosa, YES for standardized tests. No for teacher made tests, assessments, evaluations, etc. . . .
Since the highest math scores are in China and other Asian countries, maybe there is a power shift. Or there is no relationship between power and test scroes. Afterall, the power reprresented by the US is due to the influence of the .1% in the world.
Máté, my understanding is that there were other intl ranking tests around in the decades before PISA, and that we’ve always scored middling– except on the 1st one, a math comparison of 12 countries in the early ‘60’s where we came in dead last. Doesn’t look like there’s any connection between such test results & economy, which tail wagging dog, etc.
Thanks for your insights into the problems with making test scores too important and for all the erroneous assumptions about the results. You are so right about the cultural and linguistic problems in translating tests. I have seen the same issues in the New York State tests that have been translated for second language students.
Laura,
I respectfully disagree. Regardless of the imperfections of the PISA test, you’ve gotta admit they measure something vaguely math at least. And if we’re putting 100% more effort into math than we used to only to get 0% gains on these math test, that’s pretty damning about our approach, isn’t it?
My 7th graders have had more math than any generation of Americans ever. And yet today, when I returned graded history assignments to them, almost none of them could convert 43/60 into a percentage. This is a scandal.
“you’ve gotta admit they measure something vaguely math at least.”
Ummm, no one doesn’t have to admit that “they measure something”. They don’t measure anything.
The TESTS MEASURE NOTHING, quite literally when you realize what is actually happening with them. Richard Phelps, a staunch standardized test proponent (he has written at least two books defending the standardized testing malpractices) in the introduction to “Correcting Fallacies About Educational and Psychological Testing” unwittingly lets the cat out of the bag with this statement:
“Physical tests, such as those conducted by engineers, can be standardized, of course [why of course of course], but in this volume , we focus on the measurement of latent (i.e., nonobservable) mental, and not physical, traits.” [my addition]
Notice how he is trying to assert by proximity that educational standardized testing and the testing done by engineers are basically the same, in other words a “truly scientific endeavor”. The same by proximity is not a good rhetorical/debating technique.
Since there is no agreement on a standard unit of learning, there is no exemplar of that standard unit and there is no measuring device calibrated against said non-existent standard unit, how is it possible to “measure the nonobservable”?
THE TESTS MEASURE NOTHING for how is it possible to “measure” the nonobservable with a non-existing measuring device that is not calibrated against a non-existing standard unit of learning?????
PURE LOGICAL INSANITY!
The basic fallacy of this is the confusing and conflating metrological (metrology is the scientific study of measurement) measuring and measuring that connotes assessing, evaluating and judging. The two meanings are not the same and confusing and conflating them is a very easy way to make it appear that standards and standardized testing are “scientific endeavors”-objective and not subjective like assessing, evaluating and judging.
That supposedly objective results are used to justify discrimination against many students for their life circumstances and inherent intellectual traits.
Ponderosa, increased classtime doesn’t tell you anything about what’s being taught or how/ what the tests measure. When my youngest flunked the practice-math-HSPA [hisch grad test] as a jr, he was reqd to drop an elective & double up on math classes sr yr. My conclusion was only that the 2nd math class taught how to pass that exam. As an adult, he has a good common-sense grasp of numbers, which I expect he learned mainly thro hischyrs working retail, & a decade of budgeting, & doing his taxes.
I like the example you gave re: your 7th-graders. I was a lousy math student & perhaps couldn’t have seen as a 12y.o. that the Q needs to be formatted as 43/60=x/100. But yrs & yrs of budgeting, doing taxes, & a decade of comparing proposals for an engrg firm makes the Q look easy-peasy now 😉 Kids in school are being thrown new math concepts every year, & don’t get a lot of practice/ use before they move to the next one. I think the math we learn well is the math we use daily.
“Kids in school are being thrown new math concepts every year, & don’t get a lot of practice/ use before they move to the next one.”
USE?
Most kids, even the best students, will tell you that, math, as it is taught, seems mostly USELESS. That’s why most kids truly dislike math. Tricks with numbers that have little to no meaning is a tough sell.
When it comes to novice learning (i.e. 99% of what’s taught at every level), the vast majority of K to 12 students rarely leave the concrete stage. Even abstract concepts in science or math need to taught and learned at the concrete level. Failure to stress the applications (use) of arithmetic and mathematics is a failure of adults to understand children.
Rage, that makes a lot of sense. The only area of math where I excelled & didn’t struggle was geometry. I’ve tended to attribute that to visual learning style & strength in logic. Could just as easly be that geometry was very clear & concrete application of math to observable world. It wasn’t until adulthood that [very basic] algebra became useful in everyday life; those basics I understand/ use.
bethree5
Think about the nearly endless line of adults who will tell you that they either still “suck at math” or didn’t get math until 10th or 11th grade – or even later. And almost all of them will confirm what I said. Math as it is currently taught seems pointless. Now this general feeling among most students has to produce some serious guilt or cognitive dissonance when it comes to the undue emphasis placed on it (as taught). Tricky tricks with numbers that have no meaning will always be a tough sell for the vast majority of children and young adolescents.
I took algebra and geometry in high school. I have never used either.
Hopefully, kids learn math not just for their usefulness in everyday life. They do not read Homer because it’ll help them to be a more efficient accountant.
It’s another matter, that kids shouldn’t feel that the math they learned in school was pointless—as most kids certainly feel that way.
The NCTM has made no effort to make the utility of math a core idea in math instruction; it is an afterthought. Just witness how few math answers require a unit label. Common Core math has only made this worth by making the tricks with number much more tricky than they need to be. Most math teachers are unequipped to teach meaningful applications because of generally weak backgrounds in science and engineering.The push for STEM programs has done little to change this.
Most kids dislike reading, especially works like Homer, and if they did they would also find it pointless. However no one is screaming that, “students just can’t get Homer” However there is an uproar over math deficiencies.
II agree with you. Many critics and media don’t even bother looking at the measurement whatsoever. Japan, for example, has been regarded one of the top performers in the PISA test. When their scores and rank were dropped in 2003, local media and pundits denounced the then curriculum policy (‘relaxed, ‘loose’ education) for decreasing the hours of core subjects. The education ministry took it back and imposed a rigorous curriculum once again. Tests scores/rank were improved in 2009 and thereafter.
Then, this report hit another blow. All news media reported that Japan once again “dropped” the rank to 15th in reading.
This happened despite the fact the country haven’t changed anything in core curriculum since ten year ago.
Clearly, the OECD moved the goalpost by changing their gaming scheme. I was watching a local news last night, and found that some questions –so called 21st Century Skills– were designed in problem-solving/essay style. When I saw the sample question (written in Japanese), I was stunned because it requires a high tasks skill, which is overwhelming to average 15-year-olds. To me, it’s developmentally inappropriate if the education ministry expects all junior high school students to do this kind of study under current curriculum policy.
Maybe it’s true to some extent average students in Japan are struggling in literacy due to cellphone/SNS addiction. But it’s absurd to claim their basic literacy level has “declined.”
OECD falsely assumes that all nations have common platform in education system. And media simply buy into that and have clueless Japanese commentators spread the narrative. They don’t seem to know that OECD constantly makes changes in their test designs and instruments. The tests used more than 10 years ago are NOT the same as the tests used in 2019.
Falling? Declining? Such words best describe the clueless behavior of powerful regimes(OECD/Media/henchman critics) for churning out hoax after hoax throughout the years.
Ken,
Your last paragraph says it all! Thanks!
Not sure why anyone pays any attention to an organization of cranks (OECD).
Duane, you write,
“Since there is no agreement on a standard unit of learning, there is no exemplar of that standard unit and there is no measuring device calibrated against said non-existent standard unit, how is it possible to ‘measure the nonobservable’?”
When people give standardized tests, they are agreeing to use the mean score on the test as their standard unit and the test itself as the measuring device. The exemplar is any student who has a mean score because standardized scores are given in values of deviations from the mean. And what is being measured is responses on the test, which are observable. All this is neatly captured in the quip that “I.Q. is whatever it is that I.Q. tests measure.” The real objection comes down to whether whatever it is that I.Q. tests measure is a valid stand-in for “general intelligence,” which is what people want to know about. Mental faculties are not directly observable, so we substitute for them that which we can observe, and the question becomes whether that’s a legitimate move. (It clearly isn’t with regard to the mostly very vague and quite abstract “skills” on the Gates/Coleman standards bullet list for ELA. How does one make general inference-making ability from text concrete enough to be objectively measured by one or two questions on a standardized test? That’s prima facie impossible.)
People give math tests because they want to know whether other people know math. So, the question is whether a PISA math score is a valid stand-in for knowledge of math. That, I think, is the root of your objection. It can be reasonably argued that PISA math scores are affected by lots of factors other than knowledge of math, including the ability to read the word problems on the test and comfort and familiarity with test-taking and, in particular, with test-taking on computers. And, of course, outcomes for populations will be affected by lots of non-math factors, including cultural factors like dutifulness and grit and concern for the outcome, as well as socioeconomic factors like poverty, nutrition, physical and psychological health, etc.
That different cohorts of students from different countries get pretty much the same scores over time suggests that base factors other than simply math ability are very much at play, that PISA math scores are better (more valid) indicators of ZIP code, poverty, or ability to read word problems than they are of math knowledge. Sometimes, it’s an easy matter to come up with a test that validly measures the underlying knowledge or skill. It’s not difficult, for example, to find out whether students know their times table from 1 x 1 to 12 x 12. Finding out whether students have the general ability to, say, construct a proof is another matter altogether because that’s a lot less concrete.
“Sometimes, it’s an easy matter to come up with a test that validly measures the underlying knowledge or skill.”
That “sometimes” happens only when, 1) the learning standard being tested is clearly objective, 2) the standard is age/developmentally appropriate, 3) the test item was written using simple and clear and age/developmentally appropriate language, 4) the test item was written using a non-confusing format.
Common Core ELA tests had no chance to produce valid results because they met zero out of four condition required.
Spot on
This should be the first question that should be posed to students in measurement classes in education school: what is, in fact, measurable? It’s really important that they develop deeply informed skepticism about answers to that question. That these standardized tests (and the standards they purportedly test) weren’t laughed off the national and world stage long ago suggests a problem in administrator education programs that needs to be remedied. Most teachers know that the tests are a crock. Many understand that the standards are, too, But they, too, could have better understanding of just why they are.
When standardized tests are normed, the test is supposed to be less biased. We know all tests carry some level of bias despite this process. The tests based on the CCSS have NEVER been normed. They can make students fail by raising the cut score. It is a scam.
“1) the learning standard being tested is clearly objective, 2) the standard is age/developmentally appropriate,”
To hell with standards. Whatever happened to curriculum goals and/or objectives?
The term standard implies that something is capable of being measured. It is a deformers’ go to word.
“How can anyone be against having standards in the classroom, standards for behavior or learning? Kind of hard to argue against, eh! What is so wrong with holding students accountable to educational standards? Nothing, I suppose, except when the term standard is inappropriately and incorrectly used to mean one thing while purporting to signify another, in other words lacking fidelity to truth.
“When people give standardized tests, they are agreeing to use the mean score on the test as their standard unit and the test itself as the measuring device. The exemplar is any student who has a mean score because standardized scores are given in values of deviations from the mean. And what is being measured is responses on the test, which are observable.”
Let’s try this again.
“When people give standardized tests, they are agreeing to use the mean score on the test as their standard unit and the test itself as the measuring device. The exemplar is any student who has a mean score because standardized scores are given in values of deviations from the mean. And what is being measured is responses on the test, which are observable.”
So, Bob, you are saying that a statistical artifact of the scores-the mean-is what is being measured?
I’m not sure if you are being facetious with your response or not. Please clarify (before I address that particular issue).
In a related story, dairy farmers report that there has been a .01% decrease in the production of manure this past decade even as milk production has risen by 5.0%. Hog farmers have noticed the same decrease in production of manure, which is increasingly used to make methane for burning to create electricity.
I think I have found where all this excrement is going.
Good one!
That’s a Moron
When the test hits your eye
Like an old PISA pie
That’s a moron
When the scores make you drool just like a pasta fazool)
You’re a moron
(When you dance down the street with a test as your beat
You’re insane
When you walk in a dream but you know you’re not dreaming signore
Scuzza me, but you see, back in old Napoli)
You’re a moron
A moron, that’s a moron
My friend you have topped your own record.
😀 😀 best parody ever
Wonderful!
I also subscribe to Poliico’s Morning Edition newsletter. The current edition is:
“BY NICOLE GAUDIANO
Presented by the Walton Family Foundation”
and they have a couple of ads in the text of the newsletter as well:
“A message from the Walton Family Foundation:
The Walton Family Foundation’s Innovative Schools Program supports educators and entrepreneurs who open all types of schools that look and feel truly different. Are you ready to be the next education innovator? Apply now for up to $325,000 in funding to turn your ideas into action. Learn More.”
“A message from the Walton Family Foundation:
The Walton Family Foundation’s Innovative Schools Program supports educators and entrepreneurs who open all types of schools that look and feel truly different and can help students achieve at the highest level. Since 2017, 14 schools have received a total of $4 million in funding to open their doors and put students on a path to upward economic and social mobility.
Are you ready to be the next education innovator?
Complete a pre-application to be eligible for up to $325,000 in funding to turn your ideas into action. The pre-application period closes January 15, 2020. Apply Now.”
And this image, which I’m not sure I’ve pasted properly; apologies in advance:
/var/folders/hn/j1xzh16976b90xg7frzr8wbr0000gp/T/com.apple.iChat/Messages/Transfers/Image 12-4-19 at 7.16 PM.jpg
The wicked Waltons might have some takers. Sometimes my colleagues and I fantasize about starting our own charter school because we’re so mired in bullish***, much of it willingly subscribed to by our administrators, union and fellow teachers. By bulls*** I mean the ridiculous, untested, failing NGSS; the ridiculous, untested, inappropriate, failing Common Core; the vacuous test prep; social emotional education (3rd rate self-help dressed up as meaningful education); evaporating discipline; infinite special ed mandates; and the increasing lack of autonomy and professional discretion that accompanies all of the above. The best teachers are the ones who seem most eager to head for the exits. “Reform” is part of the problem, but the public school establishment is itself deeply flawed.
Ponderosa
I know you teach SS, but I’m curious about your take on the NGSS as it is actually playing out in the classroom. In my opinion, the NextGen science standards are far worse than Common Core standards in ELA. The very essence of K to 8+ science educationas has been completely corrupted. Here in NYS we are a bit behind CA; the shift in my district has been very gradual and fortunately has not affected my traditional lab program. So what are you seeing and hearing on the ground?
Rage,
What I’ve seen is science teachers more disgruntled than ever; teachers telling me not only can they not figure out how to teach the standards, they cannot even comprehend the standards; teachers complaining about the lack of materials and having to ‘build the plane while flying it’; science going from one of kids’ favorite classes to one of their least favorite classes; kids complaining to ME (not their science teacher) about their inability to understand what’s going on in science class; publishers’ sample Discovery Education NGSS “tech books” that are the driest, most unintelligible books I’ve ever seen; and dry, obtuse NGSS-aligned worksheets left on the copier machine. I once had to substitute for the 7th grade science teacher. Her warm-up exercise entailed responding to a question that made no sense to me. I groaned inwardly when I saw it. The kids couldn’t make heads or tails of it either judging from their responses. I forget the exact wording but it was one of those faux-rigorous, standards-aligned kind of questions that seems to preen itself on its wondrous sophistication and higher-orderedness. In truth it was just a bad question. At the California Teachers’ Association’s misnamed Good Teaching Conference, I sat in on a NGSS session where the very unimpressive presenter baffled us all with her opaque explanations. She spoke vaguely about students fanning out into the fields, developing their own questions, and devising experiments to discover science on their own. She had us do an activity with some manipulatives that had us all stumped. Around me I spied what seemed like sharp, veteran teachers who were clearly underwhelmed, as well as earnest “team players” who couldn’t help but admit they were deeply confused too, and knew if they were confused, their students would be even more confused. I agree with you: I think it’s even more of a disaster than CC ELA, which is saying a lot. I’m beginning to despair of public schools. We have no immunity to idiocy.
“the public school establishment is itself deeply flawed.”
“my colleagues and I fantasize about starting our own charter school ”
How is the above sound different from the usual reformi speech?
Ponderosa
I appreciate the detailed response, thanks.
New York’s NGSS were tweaked slightly, but read no better than what you have described. I really feel for the elementary teachers who will find it nearly impossible to distill the actual science content that is hidden inside the most poorly constructed and baffling “learning standards” ever written. (No exaggeration here). Unfortunately, even if they can find the content, they will discover that it is disjointed beyond all logic thanks to a scope and sequence that rejects traditional disciplines or so-called “silos” in favor of an integrated format that only adds to the confusion of the teachers and students.
If a foreign country wanted to subvert the technical expertise of our nation, they could do no worse than force the bogus and debunked “discovery” approach to science on a generation of students. Hopefully the 31 states that have rejected the NGSS have retained more traditional and successful and enjoyable science standards.
The most telling rejection of the NGSS will be by the students. Not even serious, highly motivated students want to be in a position of having to “(re)discover” the foundational facts, principles, concepts, and methods of science. NGSS is peddling failed, debunked, and re-branded methodologies, soaked with enough 21st century jargon to fool administrators, BOEs, and unsuspecting elementary school teachers.
I completely agree with you on the direction that public education has taken. It is now a system hijacked by adults who have demonstrated virtually zero knowledge and understanding about the ways that children learn, grow, and develop. Hijacked by adults who seem to believe that children and young adolescents are proto-adults who will magically learn best by “doing” without ever learning. Hijacked by adults who have re-branded confusion as “rigor”. Hijacked by adults who turned schools into centers of confusion, tension, and testing drudgery – gutted of interesting, meaningful, and engaging content knowledge. Beyond sad.