As I wrote in an earlier post, NAEP Proficient is not the same as “grade level.” NAEP Proficient is equivalent to an A or an A-. Secretary of Education Cardona made the egregious error of saying at a Congressional hearing on April 18 that 2/3 of the students in this nation were below grade level. He was wrong.
Tom Loveless, then of the Brookings Institution (now retired), wrote an excellent article in 2016, providing a history of NAEP and explaining just how high the standard for NAEP Proficient is. He was responding to the wildly inaccurate claims of rightwing ideologues, who said the same things that Secretary Cardona said.
Here are some excerpts from his article, “The NAEP Proficiency Myth.”
Equating NAEP proficiency with grade level is bogus. Indeed, the validity of the achievement levels themselves is questionable. They immediately came under fire in reviews by the U.S. Government Accountability Office, the National Academy of Sciences, and the National Academy of Education.[1] The National Academy of Sciences report was particularly scathing, labeling NAEP’s achievement levels as “fundamentally flawed.”
Despite warnings of NAEP authorities and critical reviews from scholars, some commentators, typically from advocacy groups, continue to confound NAEP proficient with grade level. Organizations that support school reform, such as Achieve Inc. and Students First, prominently misuse the term on their websites. Achieve presses states to adopt cut points aligned with NAEP proficient as part of new Common Core-based accountability systems. Achieve argues that this will inform parents whether children “can do grade level work.” No, it will not. That claim is misleading….
Today’s eighth graders have made it about half-way to NAEP proficient in 25 years, but they still need to gain almost two more years of math learning (17 points) to reach that level. And, don’t forget, that’s just the national average, so even when that lofty goal is achieved, half of the nation’s students will still fall short of proficient. Advocates of the NAEP proficient standard want it to be for allstudents. That is ridiculous. Another way to think about it: proficient for today’s eighth graders reflects approximately what the average twelfth grader knew in mathematics in 1990. Someday the average eighth grader may be able to do that level of mathematics. But it won’t be soon, and it won’t be every student.
In the 2007 Brown Center Report on American Education, I questioned whether NAEP proficient is a reasonable achievement standard.[2] That year, a study by Gary Phillips of American Institutes for Research was published that projected the 2007 TIMSS scores on the NAEP scale. Phillips posed the question: based on TIMSS, how many students in other countries would score proficient or better on NAEP? The study’s methodology only produces approximations, but they are eye-popping….
Singapore was the top scoring nation on TIMSS that year, but even there, more than a quarter of students fail to reach NAEP proficient. Japan is not usually considered a slouch on international math assessments, but 43% of its eighth graders fall short. The U.S. looks weak, with only 26% of students proficient. But England, Israel, and Italy are even weaker. Norway, a wealthy nation with per capita GDP almost twice that of the U.S., can only get 9 out of 100 eighth graders to NAEP proficient.
Finland isn’t shown in the table because it didn’t participate in the 2007 TIMSS. But it did in 2011, with Finland and the U.S. scoring about the same in eighth grade math. Had Finland’s eighth graders taken NAEP in 2011, it’s a good bet that the proportion scoring below NAEP proficient would have been similar to that in the U.S. And yet articles such as “Why Finland Has the Best Schools,” appear regularly in the U.S. press….[3]
NAEP proficient is not synonymous with grade level. NAEP officials urge that proficient not be interpreted as reflecting grade level work. It is a standard set much higher than that. Scholarly panels have reviewed the NAEP achievement standards and found them flawed. The highest scoring nations of the world would appear to be mediocre or poor performers if judged by the NAEP proficient standard. Even large numbers of U.S. calculus students fall short.
As states consider building benchmarks for student performance into accountability systems, they should not use NAEP proficient—or any standard aligned with NAEP proficient—as a benchmark. It is an unreasonable expectation, one that ill serves America’s students, parents, and teachers–and the effort to improve America’s schools.
So it seems to me NAEP could take the air out of this bag by changing “proficient” to “mastery” Although the Webster dictionary defines proficient as “highly competent”, that is not how right wingers or, for that matter, the national media, uses the term. They seem to see proficient as “good enough.” So let’s out wordsmith these neanderthals. If one has to explain justification, then you have already lost the argument.
“So let’s out wordsmith these neanderthals.”
One cannot out-wordsmith out of the onto-epistemological morass that is the standards and testing malpractice regime (of which NAEP is a part). Crap in crap out, plain and simple. Doing the wrong thing righter as per Russ Ackhoff:
Doing the Wrong Thing Righter
The proliferation of educational assessments, evaluations and canned programs belongs in the category of what systems theorist Russ Ackoff describes as “doing the wrong thing righter. The righter we do the wrong thing,” he explains, “the wronger we become. When we make a mistake doing the wrong thing and correct it, we become wronger. When we make a mistake doing the right thing and correct it, we become righter. Therefore, it is better to do the right thing wrong than the wrong thing right.”
I would be pleased to see NAEP used to replace all state assessments. It is conducted on a sampling basis. There are no reports tied to individual students, teachers, or schools. It provides a snapshot in time, scores by race, state, gender, sped, free lunch, achievement gaps, etc.
Correction: “It provides a AN INCOHERENT snapshot in time, USING INVALID scores by race, state, gender, sped, free lunch, achievement gaps, etc.”
It is a no-stakes exam. No student takes the entire exam. It has no consequences. I would love to see it replace all state assessments, which are used to stigmatize students, teachers, and schools.
Thank you for posting this.
Let’s discuss NAEP basic and poor reading instruction for the complicated language English is with 26 letters, 44 sounds and 250 symbols not including the crazy number of fonts. Finnish for example is a transparent language and requires fewer cognitive resources to decode.
In addition to their native language, most Finns speak/read English. And, a good number are also fluent in German. Finns also don’t kill a love of reading before the age of 7.
The problem with the term proficient is that it is a subjective term. When used to describe the NAEP, many including Sec. Cardona believe that it means barely reaching a benchmark.
I recall that several years ago Steven Krashen did an analysis on the PISA test when politicians balked about US student scores being in the middle of the pack. If I remember correctly, he broke the results into quintiles by socioeconomic groupings. He then showed that US students did well in each quintile when socioeconomics were a consideration. I wonder if any similar analysis has ever been done on the NAEP. results.https://dianeravitch.net/2012/12/09/stephen-krashen-our-pisa-scores-are-just-right/
I can tell you that rankings on NAEP correlate to family income, as they do on every standardized test
The trend has always been what’s important.
I have always wondered why we expect this year’s eighth graders to have higher test scores than last year’s eighth graders. Same kids of same age with mostly same teachers and same textbooks.
The reality is that what we expect kids to do on tests is far harder than it ever was. The kids are the same but the tests are far more demanding.
Go to the NAEP website and look at sample 8th grade math questions. I can’t do them. Can you?
I don’t know if the goal is to have NAEP scores increase. But it’s worth knowing if they’re increasing or decreasing, I think.
Over the past decade, since the introduction of Common Core, NAEP scores have been stagnant.
NEAP reading and math scores use a 0 – 500-point scale.
So, a 5-point drop is only a 1% decrease.
The decreases being reported are barely statistically significant and are well within the margin of error.
it should also be noted that about 100,000 students are tested in any cohort. Nationally, the fourth-grade cohort includes about 4 million students. Therefore, 97.5% of students in any tested cohort do not take the NAEP tests.
NAEP tests are also the ultimate in NO-stakes exams.
Majority of students who take them have zero incentive to try their best. NAEP testing is a joke, yet the media acts like these scores are the holy grail.
To demonstrate just how clueless they are, NAEP not only tests 4th graders and 8th graders but they absolutely waste their time and energy and money testing Seniors in high school! Talk about useless data.
Seniors treat the tests as a joke. Younger students try their best. No student takes the entire exam. NAEP uses a statistical process called Bibb sampling.
I’ve been told for years and years and years that NAEP is the “gold standard.” It’s useless?
No, it’s very useful. I think NAEP should replace all the state assessments.
Why do you think a test is good only if scores go up every year?
Why should fourth graders this year be smarter than fourth graders two years ago?
Are you asking me those questions? I didn’t say I thought those things.
I thought that was your implication. I apologize if I was wrong. The media seems to believe that scores must go up every year—as NCLB mandated—or schools are failing. Nonsense.
I think it’s worth knowing if they’re increasing, decreasing, or staying the same. Certainly they can’t keep increasing forever, just as they can’t keep decreasing forever. But I think it’s worth knowing if there are trends.
“But it’s worth knowing if they’re increasing or decreasing, I think.”
No, it’s worthless. See above comment about crap in, crap out.
We disagree about standardized tests and Noel Wilson, though, Duane.
“ I have always wondered why we expect this year’s eighth graders to have higher test scores than last year’s eighth graders. Same kids of same age with mostly same teachers and same textbooks.”
I think the expectation of progress stems from the “raise the bar and they’ll learn to jump higher” mantra that we heard so often when the CCSS were first initiated. By eighth grade, those trained seals should be able to jump to that A level.
Hasn’t worked out that way, though…has it?
I also have a theory that we’re seeing the corporate mindset, here. Profit driven companies require growth and/or, at least, reliable stability in order to satisfy and retain their investors’ confidence. If the goals are set to an unreasonably high level; the metrics for calculating progress can be rigged and profits from the corresponding materials/curriculum will follow for a longer period of time.
Beyond the confusion of NAEP achievement levels lies the 800 lb. gorilla riding the elephant around the testing room: The NAEP “frameworks” (i.e. standards) are closely aligned with the Common Core standards in ELA; standards that are not only vague, subjective, and void of any serious content (parts of speech, god forbid), but standards that completely ignore brain development and academic readiness in children and young adolescents. And in the world of test development, it is understood that a test can only be as good as the standards that drive the writing of individual items. Crappy standards will always result in crappy tests. In the case of Common Core language arts standards and their bastard spawn, not just crappy – but standards purposely written to skew scores downward; true academic death traps for all but a minority of students. And as someone who administered them all, I can attest that they do not even come close to providing an accurate assessment of basic reading skills.
Important point.
How does the “kids can’t read”, test score crowd explain data like this from a local school district:
From NYSED Data Site (School Report Cards):
Grade 4 ELA: 27% proficiency (passing)
Grade 4 Science: 87% proficiency (passing)
And this comparison is not an outlier.
That’s odd because science tests and math tests are also reading tests
It’s not odd at all. Same thing happens in 8th grade.
The reasons:
1) NYS science standards are objective, concrete and developmentally appropriate; hence the tests are fair and reasonable for grades 4 and 8. (The new NGSS standards are Common Core inspired so prepare for NY science scores to drop significantly with the new tests).
2) Reading success on any one of the three state tests depends largely on having a strong vocabulary and solid background knowledge. Science instruction provides all students with these two critical pieces; topics are also very discrete and independent. Reading passages on the ELA tests inevitably favor advantaged students from middle to upper class families because they develop more comprehensive vocabularies and accrue so much more general knowledge from outside the classroom experiences. Dinner table, conversations, vacations, museum visits, music lessons, sports teams, play groups, home libraries, etc. that all contribute to the acquisition of language skills that so many disadvantaged children miss out on.
3) Science tends to be new and more interesting to students which provides motivation on tests.
Kids CAN read, when the right pieces are in place.
Until we destroy the standards and testing malpractice regime these mental masturbation conversations will continue, hands will be wrung, the teaching and learning process bastardized. . .
. . . and the students will all continue to be harmed.
But hey, what’s a little collateral damage to the students. . .
. . . as long as I keep my job!?
Miguel Cardona is not proficient or on grade level when it comes to, if you’ll pardon, secretarying education. He’s far below basic.
All of these “measurements” and comparisons seem silly to me. What’s important is not what test scores suggest, but what politicians and astroturf groups use the scores for. Their intention is to degrade the quality of American labor. They use “lies, damn lies, and statistics” to excuse devaluing the workforce. The fact is that the value of young people does not change over time. They needn’t work longer hours. They mustn’t have their wages suppressed when they grow up. The scores are just bunk.
The scores are used to tell lies about schools, students, and teachers. Sad that Cardona is doing the Duncan bit.
I did a bit of digging earlier this week and found a description for below basic, basic, proficient, and advanced.
Basic means grade level
Proficient means above grade level
Advanced, well… it speaks for itself
Below basic, like Advanced… speaks for itself but there is no way to tell how far below basic each child is. Some could be close to BASIC, meaning grade level.
Once we look at the ratios in each category, about two-thirds of children in the US are reading at or above grade level.
Sadly, Cardona doesn’t know what you just explained, Lloyd. I wonder why.
Why?
Because he’s an adminimal by inclination and training.