Tom Loveless of the Brookings Institution has studied student achievement for many years. He has written several reports on the National Assessment of Educational Progress (NAEP). Before earning his doctorate, he taught sixth grade in California.
In this post, he explains why “reformers” who confuse NAEP’s “proficient level” with “grade level” are wrong.
This claim has been asserted by pundits like Campbell Brown of The 74, Michelle Rhee, and organizations such as Achieve. They want the public to believe that our public schools are failing miserably, and our kids are woefully dumb. But Loveless shows why they are wrong.
He writes:
Equating NAEP proficiency with grade level is bogus. Indeed, the validity of the achievement levels themselves is questionable. They immediately came under fire in reviews by the U.S. Government Accountability Office, the National Academy of Sciences, and the National Academy of Education.[1] The National Academy of Sciences report was particularly scathing, labeling NAEP’s achievement levels as “fundamentally flawed.”
Despite warnings of NAEP authorities and critical reviews from scholars, some commentators, typically from advocacy groups, continue to confound NAEP proficient with grade level. Organizations that support school reform, such as Achieve Inc. and Students First, prominently misuse the term on their websites. Achieve presses states to adopt cut points aligned with NAEP proficient as part of new Common Core-based accountability systems. Achieve argues that this will inform parents whether children “can do grade level work.” No, it will not. That claim is misleading.
The expectation that all students might one day reach 100% proficiency on the NAEP is completely unrealistic. It has not happened in any other country, including the highest performing. Not even our very top students taught by our very best teachers haven’t reached 100% proficiency. This is a myth that should be discarded.
Loveless goes even farther and insists that NAEP achievement levels should not be the benchmark for student progress.
He warns:
Confounding NAEP proficient with grade-level is uninformed. Designating NAEP proficient as the achievement benchmark for accountability systems is certainly not cautious use. If high school students are required to meet NAEP proficient to graduate from high school, large numbers will fail. If middle and elementary school students are forced to repeat grades because they fall short of a standard anchored to NAEP proficient, vast numbers will repeat grades.
Anyone who claims that NAEP proficient is the same as grade level should not be taken seriously. Loveless doesn’t point out that the designers of the Common Core tests decided to align their “passing mark” with NAEP proficient, which explains why 70% of students typically fails the PARCC and the SBAC tests. Bear in mind that the passing mark (the cut score) can be arbitrarily set anywhere–so that all the students “pass,” no students pass, or some set percentage will pass. That’s because the questions have been pre-tested, and test developers know their level of difficulty. And that is why U.S. Secretary of Education John King, when he was New York Commissioner of Education, predicted that only 30% of the students who took the state tests would “pass.” He was uncannily accurate because he already knew that the test was designed to “fail” 70%.
He concludes:
NAEP proficient is not synonymous with grade level. NAEP officials urge that proficient not be interpreted as reflecting grade level work. It is a standard set much higher than that. Scholarly panels have reviewed the NAEP achievement standards and found them flawed. The highest scoring nations of the world would appear to be mediocre or poor performers if judged by the NAEP proficient standard. Even large numbers of U.S. calculus students fall short.
As states consider building benchmarks for student performance into accountability systems, they should not use NAEP proficient—or any standard aligned with NAEP proficient—as a benchmark. It is an unreasonable expectation, one that ill serves America’s students, parents, and teachers–and the effort to improve America’s schools.
Diane Ravitch's blog 
No effort to summarize an individual with a score is valid about that individual. The usefulness of any mass data is limited.
LikeLiked by 1 person
I agree.
I go back to this letter by Lois Levy, a private school administrator, written to the parents of the students at her school. She carefully points out the downside and limits to standardized testing, and how they are administered reluctantly at her school, and how grateful she is that, unlike in public schools, the leaders at her school “aren’t forced into the position of using test scores as the dominant means of evaluating our curriculum (and the quality of the teachers, school, etc. JACK).”
None of the upscale private schools catering to the wealthy use Common Core testing and curriculum. Instead, they use ERB, a sort of standardized testing LITE for elite private schools, radically different from the Common Core version that is mandated for and forced upon the public school system. (It’s the one used at Heschel, where Campbell Brown and her husband Dan Senor sends their two boys.)
Here’s that letter from Lois Levy, an administrator (“Assistant Head of School”) at a different private school, the Center for Early Education (CEE), describing “what ERB standardized testing is and isn’t.”
http://www.centerforearlyeducation.org/page.cfm?p=774&eid=559
Lois Levy says CEE subjects students to standardized tests reluctantly. It’s part of the world we live in NOW, so the folks at her school can’t NOT do it. (This is likely the same view that the folks running Heshel and other elite private schools share.)
—————————————
LOIS LEVY: “A major reason for administering the ERBs is that they provide our students with practice taking standardized tests. Whether we like it or not, standardized tests are currently a part of the educational world, mainly due to the fact that they provide an efficient way to produce data.”
—————————————
Levy minces no words about the limitations of ERB standardized tests, and standardized testing in general.
—————————————
LOIS LEVY: “Before receiving student results that will be mailed home early in the summer, it is important for parents to reflect on what these test results represent as well as what they don’t represent.
“Unlike today’s public schools, CEE is lucky in that we aren’t forced into the position of using test scores as the dominant means of evaluating our curriculum (and the quality of the teachers, school, etc. JACK).
“Instead, we are able to use our mission statement and school philosophy to guide our development of curriculum. Evaluating the effectiveness of our curriculum relies on feedback from our standardized testing program, but also on feedback from other means of assessment as well as from faculty, administration, parents, and CAIS Accreditation teams.
“It is important to first understand what ERB test results do not tell us about our students.
“ERB tests do not even attempt to measure a student’s initiative, motivation or ability to persevere.
“ERBs do not measure study skills, organizational skills, collaborative skills, cooperative skills, communication skills or creativity.
“This list could go on and on, but it is obvious that all of the above skills are essential for being a successful student, and, for that matter, a successful adult. Yet standardized tests are not able to measure these essential learning skills/life skills.
“Thus, when looking at children’s standardized test results, we always need to remember that they are not a summary of all qualities that are needed to be a successful student.”
————–
Amen, sister!!!! Testify!!! Testify!!!
She blows apart the test prep strategies that Eva and Campbell force upon the kids of the 99%. (i.e. the administrator at SUCCESS ACADEMY who proudly bragged to New York Magazine that SUCCESS ACADEMY employs marathon test prep sessions that turns its students into “little test-taking machines.”)
Levy contends that such an insane amount of tutoring and/or test prep itself nullifies the very purpose of testing — assuming one accepts that there is such a purpose — and renders the results meaningless.
—————————————
LOIS LEVY : “(Test prep) Tutoring actually defeats the purpose for what the school hopes to learn from the tests. We want to know how the curriculum supports children’s learning, not how tutoring may or may not bolster results.
“For many years, tutoring for ERBs was not even an issue as it was understood that the ERBs are not ‘high stakes’ tests; they are not admissions tests. But as parents get more concerned about how well their young children ‘measure up’ against other children, fear and anxieties can drive parents to tutoring.
“Test prep is a huge business and professional tutors are more than happy to take parents’ money, especially since their work preparing children for admissions tests end in December or January.”
” .. ”
“In summary, my goal is to help parents understand what ERB test results do measure, and what they don’t measure. When parents receive the one sheet of paper summarizing their child’s ERB test results, it is important for them to remember that these results don’t point to any hard cold facts about their child.
“Instead, the results need to be viewed in conjunction with the child’s classroom performance, the child’s developmental learning path, as well as teacher and parent observations.
“Standardized tests are indirect measurement tools that only measure how a child performed on a given day and on a given set of questions.”
—————————————
This points out the stark difference between what version of standardized testing and its accompanying emphases & importance as they are administered at Heschel and other elite private schools… and what Campbell and other corporate reformers want for the children of the middle and working classes in her Brave New Privatized School World vs. what they want for their own children.
Do you get it? The difference between…
… what Campbell Brown wants for her own children, and for the other children of the 1%—
VERSUS
… what Campbell Brown wants for “other people’s children” of the 99%, should she achieve her David Cameron wet dream of eliminating all public schools (that currently have oversight and input from the public via democratically elected school boards) and replacing them with privately-managed charters with ZERO accountability to the public, ZERO transparency to the public, and which do not educate all the public… basically private schools with public money ?
And here’s Matt Famer, quoting Dr. David Magill, the leader at the Chicago Lab School (where Obama sent his kids before moving to D.C., and where corporate reform Mayor Rahm Emanuel and recently departed Secretary of Ed. Arne Duncan currently send their kids… and which has no Common Core testing or curriculum whatsoever.):
(4:54 –
(4:54 –
————————————–
DR. DAVID MAGILL, DIRECTOR OF THE CHICAGO LAB SCHOOL (from the Chicago Lab School website:
“Measuring outcomes of standardized testing, and referring to those results as the evidence of learning, and the bottom line is, in my opinion, misguided, and unfortunately continues to be advocated under a new name, and supported by the current (U.S.? Chicago Public schools?… not sure…) administration.”
————————————–
LikeLike
“No effort to summarize an individual with a score is valid about that individual.”
I think I understand your statement but would you please clarify/expand and explain? TIA, Duane
LikeLike
Jack: much appreciate your contribution to this thread.
If I may, I believe you are referencing a $ucce$$ Academy piece from NYMAGAZINE, 4-25-2010. Title: “The Patron Saint (and Scourge) of Lost Schools.” Subtitle: “Eva Moskowitz, the controversial leader of the fastest-growing charter network in the city, wants to save New York public education by, in a sense, destroying it.”
[start excerpt]
The day before the scheduled math test, the city got socked with eight inches of snow. Of 1,499 schools in the city, 1,498 were closed. But at Harlem Success Academy 1, 50-odd third-graders trudged through 35-mile-per-hour gusts for a four-hour session over Subway sandwiches. As Moskowitz told the Times, “I was ready to come in this morning and crank the heating boilers myself if I had to.”
“We have a gap to close, so I want the kids on edge, constantly,” Fucaloro adds. “By the time test day came, they were like little test-taking machines.”
[end excerpt]
Link: http://nymag.com/nymag/features/65614/
And just who is being quoted? From the above linked article: “Paul Fucaloro, her director of instruction and right-hand man.”
I urge readers of this blog to read this six-year-old piece—it is still relevant.
Of course, I advise one and all that if they should encounter steroid-using road-rage six-year-olds in a $ucce$$ Academy classroom, I urge them to Dial up their courage and get ahold of that Got-to-Go list so they aren’t the victims of the heavy desks these dangerous felons may toss at them.
😱
Forewarned is forearmed.
😎
LikeLike
How many times does one have to explain to someone like Brown that ‘NAEP proficiency” is not the same as ‘grade level” before one concludes that she is not simply acting out of ignorance? (ie, that it is not merely “confusion” that makes her say what she does)
The way that Campbell Brown elided from her original talk of “Proficiency” (before she was criticized) to her talk of “Grade level proficiency” gave away her game.
In my opinion, “confusion” (of the public) is actually the goal.
LikeLiked by 1 person
Poet,
She wants the public to believe that public education is a failure. This is one of her talking points.
LikeLike
It goes way beyond the mere confusion of scoring on NAEP and what proficient either does or should mean. In this middling time of my teaching career (15-ish years) standardized tests have gone from bearing the disclaimer that results should not be the only determining factor in programming decisions or judgments regarding academic progress, to being the preferred tools of attack for elites birthed out of privilege, private schools, or empowered to bear the water and craft the narrative for those folks use against my profession. Explicitly. With intent. Warped and twisted of definition to purposely support an agenda of ignoring the deeper endemic social, political and economic illnesses that benefit and protect those very elites attacking public education. Ignoring that while they rest comfortably and well fed, able to continue choosing elite schools for their children, enriching experiences, a wide smooth road and maybe a trust fund to go with it…They are suggesting segregation, oppression, and disempowerment via data for the classes and children below.
All the while the agenda is being packaged for sale, as choice and equity, and the people bringing it as the caring missionaries for college and career ready test scores.
LikeLike
“And that is why U.S. Secretary of Education John King, when he was New York Commissioner of Education, predicted that only 70% of the students who took the state tests would “pass.” He was uncannily accurate because he already knew that the test was designed to ‘fail’ 70%.”
This does not make sense to me– if 70% pass, wouldn’t 30% fail?
LikeLike
My error
I will fix it
LikeLike
I read carefully the brookings article referred to. I was left with a question which I think is unanswered: why did NAEP use proficient as a word when they really meant very advanced as compared to their peers? Do we get any news about our calculus students fromNAEP? Are there questions on there about derivatives and rates of change? What is it that we want out of this and other tests?
LikeLike