Search results for: "NAEP"
« Older Entries

Why NAEP Scores in History and Civics Are Meaningless in These Times

By dianeravitch
May 22, 2023 //
47

I served on the governing board of the National Assessment of Educational Progress (NAEP) for seven years. I was appointed by President Clinton. I learned quite a lot about standardized testing during that time. I enjoyed reading test questions and finding a few that had two right answers. Two subjects where I felt confident as a reviewer, in addition to reading, were history and civics.

I was momentarily dismayed, but not surprised, to learn that the NAEP scores in history and civics had declined, as they had in reading and math, after the disruptions and closings caused by the pandemic. This is not surprising, because fewer days of instruction translates into less learning.

So we know for sure that instructional time matters. You can’t learn what you weren’t taught.

But on second thought, I realized that in these days it is almost impossible to test history and civics and get a meaningful result.

Many states, all Republican-dominated, have censored history teaching. The legislatures don’t want students to learn “divisive concepts.” They don’t want anything taught that will make students “uncomfortable.” They don’t want “critical race theory” to be taught. These ideas have been spun out at length with other vague descriptions of what teachers are NOT allowed to teach.

The people who write test questions for NAEP history are not bound by these restrictions. They are most likely writing questions about “divisive concepts” and “uncomfortable” topics. They might even ask questions that legislators might think are tinged or saturated by critical race theory.

Given the number of states that ban the teaching of accurate, factual history, it’s seems to me impossible to expect students to be prepared to take an American history test.

Even more complicated is civics. A good civics exam might ask questions about the importance of the right to vote. It might ask questions written on the assumption that vote suppression and gerrymandering are undemocratic practices that were long ago banned by the courts. Yet courts are now allowing these baleful practices to stand. How can a student understand that a discredited practice is now openly endorsed in various state laws and have not been discredited by the courts?

Civics classes typically teach that one of the great strengths of American democracy is the peaceful transition of power from one President to another. How can they teach that idea when Trump partisans insist that he won the last election and was ousted in a coup? How can teachers explain the election process when Trump says it’s rigged (he said it before the 2016 election as well)? How can students answer questions about elections and the Electoral College when Trumpers believe they were corrupted in 2020?

How can teachers teach civics when almost every GOP leader asserts that the election was stolen?

How can civics be taught when public officials defy public opinion to allow any individual to buy guns without a background check or a permit. Having bought a gun, they may wear it openly in some states and carry it concealed in some other states. Students have been practicing in case an armed killer walks into their school during the day. They need only google to learn that a majority of the public favors gun control of varying kinds. Why, they might ask their teacher, doesn’t the legislature and Congress act to protect the lives of children?

Is it worse to teach lies or to teach the truth?

Categories Accountability, Censorship, Civics, Corruption, Curriculum, Elections, Extremism, History, Lies, NAEP, Racism, Religion, Segregation, Separation of church and state, Standardized Testing, Students, Teachers and Teaching

Tom Loveless: Why NAEP “Proficient” is a High Standard, Not “Grade Level”

By dianeravitch
April 20, 2023 //
35

As I wrote in an earlier post, NAEP Proficient is not the same as “grade level.” NAEP Proficient is equivalent to an A or an A-. Secretary of Education Cardona made the egregious error of saying at a Congressional hearing on April 18 that 2/3 of the students in this nation were below grade level. He was wrong.

Tom Loveless, then of the Brookings Institution (now retired), wrote an excellent article in 2016, providing a history of NAEP and explaining just how high the standard for NAEP Proficient is. He was responding to the wildly inaccurate claims of rightwing ideologues, who said the same things that Secretary Cardona said.

Here are some excerpts from his article, “The NAEP Proficiency Myth.”

Equating NAEP proficiency with grade level is bogus. Indeed, the validity of the achievement levels themselves is questionable. They immediately came under fire in reviews by the U.S. Government Accountability Office, the National Academy of Sciences, and the National Academy of Education.[1] The National Academy of Sciences report was particularly scathing, labeling NAEP’s achievement levels as “fundamentally flawed.”

Despite warnings of NAEP authorities and critical reviews from scholars, some commentators, typically from advocacy groups, continue to confound NAEP proficient with grade level. Organizations that support school reform, such as Achieve Inc. and Students First, prominently misuse the term on their websites. Achieve presses states to adopt cut points aligned with NAEP proficient as part of new Common Core-based accountability systems. Achieve argues that this will inform parents whether children “can do grade level work.” No, it will not. That claim is misleading….

Today’s eighth graders have made it about half-way to NAEP proficient in 25 years, but they still need to gain almost two more years of math learning (17 points) to reach that level. And, don’t forget, that’s just the national average, so even when that lofty goal is achieved, half of the nation’s students will still fall short of proficient. Advocates of the NAEP proficient standard want it to be for allstudents. That is ridiculous. Another way to think about it: proficient for today’s eighth graders reflects approximately what the average twelfth grader knew in mathematics in 1990. Someday the average eighth grader may be able to do that level of mathematics. But it won’t be soon, and it won’t be every student.

In the 2007 Brown Center Report on American Education, I questioned whether NAEP proficient is a reasonable achievement standard.[2] That year, a study by Gary Phillips of American Institutes for Research was published that projected the 2007 TIMSS scores on the NAEP scale. Phillips posed the question: based on TIMSS, how many students in other countries would score proficient or better on NAEP? The study’s methodology only produces approximations, but they are eye-popping….

Singapore was the top scoring nation on TIMSS that year, but even there, more than a quarter of students fail to reach NAEP proficient. Japan is not usually considered a slouch on international math assessments, but 43% of its eighth graders fall short. The U.S. looks weak, with only 26% of students proficient. But England, Israel, and Italy are even weaker. Norway, a wealthy nation with per capita GDP almost twice that of the U.S., can only get 9 out of 100 eighth graders to NAEP proficient.

Finland isn’t shown in the table because it didn’t participate in the 2007 TIMSS. But it did in 2011, with Finland and the U.S. scoring about the same in eighth grade math. Had Finland’s eighth graders taken NAEP in 2011, it’s a good bet that the proportion scoring below NAEP proficient would have been similar to that in the U.S. And yet articles such as “Why Finland Has the Best Schools,” appear regularly in the U.S. press….[3]

NAEP proficient is not synonymous with grade level. NAEP officials urge that proficient not be interpreted as reflecting grade level work. It is a standard set much higher than that. Scholarly panels have reviewed the NAEP achievement standards and found them flawed. The highest scoring nations of the world would appear to be mediocre or poor performers if judged by the NAEP proficient standard. Even large numbers of U.S. calculus students fall short.

As states consider building benchmarks for student performance into accountability systems, they should not use NAEP proficient—or any standard aligned with NAEP proficient—as a benchmark. It is an unreasonable expectation, one that ill serves America’s students, parents, and teachers–and the effort to improve America’s schools.

Categories Accountability, Cardona, Miguel, Data, Education Industry, Extremism, Fake, NAEP, Standardized Testing

Secretary Cardona, NAEP Proficient Is NOT Grade Level!

By dianeravitch
April 19, 2023 //
45

For the past dozen years, since the attack on public schools went into high gear, the same lie has been trotted out again and again to defame public schools. The slanderers say that 2/3 of American students are reading “below grade level.”

At Congressional hearings on the education budget on Tuesday April 18, the same ridiculous claim was made by U.S. Secretary of Education Miguel Cardona. He said that only 33% are reading at proficiency. He said this is “appalling and not acceptable for the United States. 33% of our students are reading on Grade level.” (At about 45:00).

This is nonsense. Its’s frankly appalling to hear Secretary Cardona repeating the lie spread by rightwing public school haters. He really should be briefed by officials from the National Assessment Governing Board before he testifies again.

On the NAEP (National Assessment of Educationsl Progress) tests, “proficient” does not represent grade level. Proficient is a high bar. Although the federal testing agency does not equate its achievement levels to letter grades, I would estimate (based on my seven years of experience as a member of the NAEP Governing Board) that “proficient” is about the same as an A or an A-. Do we really expect that every student merits an A? I don’t think so.

The website of the National Center on Education Statistics states clearly:

Achievement Levels

NAEP student achievement levels are performance standards that describe what students should know and be able to do. Results are reported as percentages of students performing at or above three NAEP achievement levels (NAEP Basic, NAEP Proficient, and NAEP Advanced). Students performing at or above the NAEP Proficient level on NAEP assessments demonstrate solid academic performance and competency over challenging subject matter. It should be noted that the NAEP Proficient achievement level does not represent grade level proficiency as determined by other assessment standards (e.g., state or district assessments).

Could it be any plainer? Students who score at or above NAEP Proficent “demonstrate solid academic performance and competency over challenging subject matter.” Furthermore, the NAEP Proficient level “does not represent grade level proficiency.”

Would someone please tell Secretary Cardona? When he repeats the lies of the rightwing propagandists, he maligns every teacher and student in the nation.

Someone should also inform Secretary Cardona that the NAEP achievement levels are set by panels of educators and non-educators; as such, they are subjective judgments. They have been used on a trial basis for 30 years without getting definitive clearance by testing experts commissioned by Congress to review their validity. “The latest evaluation of the NAEP achievement levels was conducted by a committee convened by the National Academies of Sciences, Engineering, and Medicine in 2016. The evaluation concluded that further evidence should be gathered to determine whether the NAEP achievement levels are reasonable, valid, and informative. Accordingly, the NCES commissioner determined that the trial status of the NAEP achievement levels should be maintained at this time.”

Please, Secretary Cardona, stop saying that “only 33% of American students can read proficiently” and that “only 33% are reading at grade level.”

It’s not true.

When asked about vouchers, Secretary Cardona said he opposes them because they take money away from public schools. That’s true, but far from the whole truth. 75-80% of vouchers subsidize students who already attend private schools. They are a transfer from the public to the affluent. Kids who leave public schools to use vouchers lose academic ground, and most return to their public school within two-three years in need of help catching up. Vouchers fund religious schools that may discriminate against students, families, and staff who do not share their religion or who are gay or who have disabilities. They choose the students they want.


Furthermore, religious schools indoctrinate. Some religious schools teach fake science and history. Religious schools force taxpayers to pay for religious views they do not share.

There are many reasons to oppose vouchers but Secretary Cardona seems unaware of them. I recommend that he invite veteran voucher researcher Joshua Cowen of Michigan State University to brief him on why vouchers for religious and private schools are a pernicious and ineffective policy.

Categories Data, Education Reform, Fake News, Fraud, Hoax, Lies, NAEP, Propaganda, Rhee, Michelle, Standardized Testing

Billy Townsend: Florida’s NAEP Hoax Again

By dianeravitch
October 31, 2022 //
11

Billy Townsend, Florida blogger, has reported regularly on Florida’s gaming of NAEP scores. He writes here that Governor Ron DeSantis is carrying out Jeb Bush’s old trick to inflate 4th grade NAEP scores. He calls the governor Ron Jebsantis. The trick is third grade retention, which ensures that the lowest scoring third graders never take the fourth grade NAEP test (the kids who take the NAEP test are selected at random).

Thus, DeSantis put out a flashy press release celebrating fourth grade NAEP scores in the test scores recently released. But, as usual, DeSantis neglects to mention the collapse of eighth grade NAEP scores. Somehow the kids who were retained in third grade managed to skip fourth grade and rejoin their classmates by eighth grade.

Here are his numbers, drawn from NAEP reports:

With that in mind, here is a view of Florida’s 2022 NAEP scores peaking in elementary school and dramatically worsening with the older cohorts —- which is ALL of the red numbers after the green baseline.

I personally put no stock in the twelfth grade numbers (which Billy extrapolated) because NAEP stopped testing seniors a decade ago. Seniors know that NAEP doesn’t count and they don’t do their best. Some don’t even try. Their answer sheets had doodles, or some just picked the (A) answer to every question or some were blank.

But the stark drop from fourth grade to eighth grade says something’s fishy in Florida.

Categories Accountability, Bush, Jeb, DeSantis, Education Industry, Failure, Florida, Hoax, NAEP

Jan Resseger: Don’t Get Stressed About the NAEP Scores

By dianeravitch
October 30, 2022 //
12

Since the two sets of NAEP scores were released recently, commentators have gone into a panic about “learning loss” and used the declines to promote their favorite reform: more of this, less of that. DeSantis even released a press release claiming falsely that Florida’s formula of ignoring the pandemic was just right (California stuck with the CDC guidelines and did at least as well, maybe better, than Florida, but Gavin Newsom did not issue a press release).

Jan Resseger has words of perspective that I sum up as: why are we surprised that learning was disrupted by the pandemic?

My question, having served on the NAEP board for seven years, is why the media and the reform crowd thinks that NAEP scores should go up every year? Why should fourth and eighth graders this year know more than fourth and eighth graders two years ago or four years ago? Isn’t it reasonable to assume that students of the same age and grade are likely to have the same scores? Yet if they do, the media sends out loud lamentations that scores are “flat.” Oh, woe! Surely we want to see a rise in the scores of the lowest scoring students, and a narrowing of gaps, but the media assumes that everyone must increase their scores or the education system is failing. This is nuts. There is little or no relationship between the test scores of students in fourth and eighth grades and the economy of the future.

Jan Resseger writes:

Are the new National Assessment of Educational Progress scores a catastrophic indication that the U.S. public schools have fallen into decline? I don’t think so.

Early this week, the National Center for Education Statistics (NCES) released a large data set from National Assessment of Educational Progress exams administered last spring to 4th and 8th grade students in U.S. public schools. Last month, NCES released scores from tests administered to a smaller group of 4th graders. Both sets of scores show that the COVID pandemic seriously disrupted schooling for the nation’s children and adolescents.

Chalkbeat’s Matt Barnum explainswhat the National Assessment of Educational Progress (NAEP) is: “The National Assessment of Educational Progress (NAEP, pronounced nape) is a test administered by an arm of the U.S. Department of Education. It’s given periodically to a representative subset of American students in math and reading in grades four and eight. Scores are broken down by state and for a select handful of cities, too. The latest results are based on tests given between January and March 2022. The previous test was given in 2019, before the pandemic… Scores from a separate NAEP exam that has been given to 9-year-olds for many decades were previously released in September.”

The NAEP scores released this week were precipitously lower than scores on the NAEP when it was administered in 2019, before COVID—particularly in 8th grade math. The Washington Post’s Laura Meckler reports: “The portion of eighth-graders rated proficient or better in math fell to 27 percent, from 34 percent in 2019… the steepest decline in more than a half century of testing.” (The fact that every year relatively few students reach NAEP’s proficient level overall is because the NAEP “proficient” cut score is set artificially high; it marks what most people would define as “advanced.”)

Some people assume that this year’s drop in NAEP scores signals a reversal of progress, the beginning of a downward spiral. Others are using the scores as evidence for their particular reform or as evidence that their state had a better policy on school closures than other states. Meckler writes: “Partisans on all sides of the education debate seized on the results to advance competing ideas about the way ahead… The test results also offered fodder for those who argue bringing students back to campuses quickly was the right move… ‘We kept schools open in 2020, and today’s NAEP results once again prove we made the right decision,’ Florida Gov. Ron DeSantis (R) said on Twitter. But the data did not establish a connection between back-to-school policies and academic performance. In California, for instance, many public schools were closed well into the 2020-21 school year and some students never saw a classroom that year. But the declines were similar to those in Texas and Florida, where schools were ordered to reopen much sooner.”

In a blog post last month when the first set of 4th grade NAEP scores was released, I shared my own assessment of what had happened. I think the scores released last month and the scores released this week show the same thing. Here is some of what I said in that post.

***

There is no cause for panic. Schooling was utterly disrupted for the nation’s children and adolescents, just as all of our lives were interrupted in so many immeasurable ways. During COVID, while some of us have experienced the catastrophic death of loved ones, all of us have experienced less definable losses—things we cannot name.

I think this year’s NAEP scores—considerably lower than pre-pandemic scores—should be understood as a marker that helps us define the magnitude of the disruption for our children during this time of COVID. The losses are academic, emotional, and social, and they all make learning harder.

Schools shut down and began remote instruction in the spring of 2020, and many stayed online through the first half of last school year. While most public schools were up and running by last spring, there have been a lot of problems—with more absences, fighting and disruption, and overwhelming stress for educators. It is clear from the disparities in the scores released last week among high and low achievers that the disruption meant very different things to different children. It is also evident that the pandemic was a jolting shock to our society’s largest civic institution. It should be no surprise, then, that the attempt to get school back on track was so rocky all through last spring…

While the NAEP is traditionally used to gauge the trajectory of overall educational achievement over time, and while the trajectory has been moderately positive over the decades, the results released last week cannot by any means be interpreted to mean a change of the overall direction of educational achievement.

Education Week’s Sarah Schwartz asked Stanford University professor Sean Reardon (whose research tracks the connection of poverty and race to educational achievement) whether “it will take another 20 years to raise scores once again.” Reardon responded: “That’s the wrong question…. The question is: What’s going to happen for these (9-year-old) kids over the next years of their lives.” Schwartz describes more of Reardon’s response: “Children born now will, hopefully, attend school without the kinds of major, national disruptions that children who were in school during the pandemic faced. Most likely, scores for 9-year-olds, will be back to normal relatively soon, Reardon said. Instead, he said, we should look to future scores for 13-year-olds, which will present a better sense of how much ground these current students have gained.”

Schwartz reports: “Students at all levels lost ground during the past two years, but lower-performing students saw the biggest drops.” The test does not in any way measure the factors that contributed to the drop in scores for students who were already struggling, but the results shouldn’t be surprising. Some children live in families with internet access and enough computers that each of several children in the family could access online instruction simultaneously, while other children’s parents had to drive them to public library or fast food outlet parking lots to find any internet access at all. Some parents had sufficient time at home to supervise children and provide assistance during online instruction, while in other families, older siblings supervised younger siblings while trying to participate themselves in online instruction. Some children and adolescents simply checked out and neglected to log-on.

Categories Accountability, Education Reform, NAEP

NAEP Hysteria at MSNBC

By dianeravitch
October 30, 2022 //
22

H. Hurley, a reader of the blog, left the following comment, which places NAEP hysteria into context:

The cherry on the journalistic cup cake related to recent NAEP reporting was an interview by Stephanie Ruhle on her 11:00 pm MSNBC program where she rushed in, of all people, ARNE DUNCAN, to discuss the CRISIS OF THE DROPPING NAEP SCORES. Her URGENCY in her set-up and interview was almost reported as a 3 alarm fire. Poor Arne. He actually tried to calm her reactions. But her hysteria is typical related to student test scores.


Nuts!


It’s obvious to real educators that a pandemic, million COVID deaths, ZOOM schooling, kids alone at home, banning books, masking, vaccing…anti vaccing, limited computer/Internet access, Jan6, school shootings, politics, chaos everywhere….shall we go on?


On top of this craziness, when children are finally returning to school, we TEST. We test & react in horror that children didn’t know the grade level content or skills. Scores dropped….who knew? Who could have predicted that?


ACTUALLY…….Anybody with some sense!
Children living in war, migration, fleeing, homeless, famine, rising fascism, massive crime, poverty, lead poisoning, hunger, job losses, craziness, etc…..are then tested under the WORST CONDITIONS.
Meanwhile, journalists hold up those results as if our children were living under heat lamps in incubators to be educated under the best conditions.


Stop the testing madness, end poverty, stop the political madness, allow families to raise their children with proper wages, fund schools, stop destroying public schools & use the election spending zillion$ on real people for a healthy nation.


My 2¢ worth!

Categories Disruption, Education Reform, Extremism, Fraud, Ignorance, NAEP, Standardized Testing

Peter Greene: Your Handy All-Purpose NAEP Press Release

By dianeravitch
October 26, 2022 //
28

Peter Greene wants to save time for all organizations that react to the latest NAEP scores. His press release works whether scores are up, down, or flat.

He writes:

It’s time once again to greet the release of another set of data from the NAEP testing machine, which means everyone is warming up their Hot Take generator. But if, like me, you’re getting tired of writing a response to the latest NAEPery, here’s a handy news release that will let you mad lib your way to NAEPy wisdom.


The new scores from the National Assessment of Education Progress (NAEP), known as The Nation’s Report Card, have been released, providing important data about [insert your preferred education policy area]. The recent crisis in [select your favorite policy-adjacent crisis] has clearly created a burgeoning issue of [select whatever Bad Thing you feel will most scare your audience in the direction of your preferred policy].

Says [head of your organization], “The new scores provide important evidence that now is the time for [insert whatever policy action your group always supports]. Clearly the [rise/drop/stagnation] in scores among [whichever subgroup cherry picking best suits your point] proves exactly what we have been arguing for [however long you’ve been at this.]”

[Insert paragraph of data carefully selected and crunched for your purposes. Add a graph if you like. People really dig graphs.]

“This is a clear indication,” says [your favorite go-to education expert], “that it is long past time to [do that thing your organization has been trying to get people to do for years]. Clearly [our preferred solution] is needed.” [Insert further sales pitch here as needed.]

You can expand on this if you wish, but make sure that you definitely do not–

* provide context for the data that you include

* offer perspective from NAEP’s many critics

* absolutely never ever reference the fact that the NAEP folks are extraordinarily clear that folks should not try to suggest a causal relationship between scores and anything else.

As always, the main lesson of NAEP is that contrary to the expectations of so many policy wonks, cold hard data does not actually solve a thing.

The NAEP remains a data-rich Rorschach test that tells us far more about the people interpreting the data than it does about the people from whom the data was collected. Button up your overcoat, prepare for greater-than-usual pearl-clutching and solution-pitching from all the folks who still think the pandemic shutdown is a great opportunity to do [whatever it is they have already been trying to do].

Categories Accountability, Data, Education Industry, Education Reform, Fake News, NAEP, Propaganda

Leonie Haimson Reviews NAEP Scores, from a NYC Perspective

By dianeravitch
October 25, 2022 //
29

The National Assessment of Educational Progress released its report on student progress in math and reading for grades 4 and 8. As expected, scores plummeted due to the many disruptions in students’ lives during the pandemic. Schools closed, then opened. Some never closed. Some were online. Some were both in-person and online. Some students lost family members to the COVID. Some students had COVID. Some teachers died or got COVID. Many cities and towns closed down. There was not a right way or a wrong way. There were many people trying to figure out what was the right thing to do. It’s still not clear, although I personally think that vaccines and masks saved many lives and reduced the seriousness of the disease for those who got it.

Leonie Haimson reviewed the results here. She provides links to other cities and states.

The Commissioner of the National Center for Education Statistics, Peggy Carr, said this:

“There’s nothing in this data that tells us that there is a measurable difference in the performance between states and districts based solely on how long schools were closed,” NCES Commissioner Peggy Carr said.

I knew Peggy Carr when I served in the US Department of Education in the early 1990s. She is a career official and a straight-talker.

Categories Accountability, Health, NAEP, Standardized Testing

Jan Resseger: NAEP Scores Show the Toll of the Pandemic—But It’s Temporary

By dianeravitch
September 12, 2022 //
57

Jan Resseger is consistently the voice of wisdom on anything related to children and young people. In this post, she explains why we should not be panicked by the decline of NAEP scores. The scores reflected the toll that the pandemic exacted. But now that children are back in school, we can expect learning to proceed without major disruption.

She writes:

I think this year’s NAEP scores—considerably lower than pre-pandemic scores—should be understood as a marker that helps us define the magnitude of the disruption for our children during this time of COVID. The losses are academic, emotional, and social, and they all make learning harder….

Education Week’s Sarah Schwartz asked Stanford University professor Sean Reardon (whose research tracks the connection of poverty and race to educational achievement) whether “it will take another 20 years to raise scores once again.” Reardon responded: “That’s the wrong question…. The question is: What’s going to happen for these (9-year-old) kids over the next years of their lives.” Schwartz describes more of Reardon’s response: “Children born now will, hopefully, attend school without the kinds of major, national disruptions that children who were in school during the pandemic faced. Most likely, scores for 9-year-olds, will be back to normal relatively soon, Reardon said. Instead, he said, we should look to future scores for 13-year-olds, which will present a better sense of how much ground these current students have gained.”

Categories Accountability, Education Reform, Health, NAEP, Testing

Leonie Haimson: NAEP Score Declines Unleash Wave of False Prescriptions

By dianeravitch
September 11, 2022 //
15

Leonie Haimson looked closely at the score declines on the National Assessment of Educational Progress and was disappointed to see the outpouring of false prescriptions. She was critical of claims that students needed to make up for lost time by being subjected to longer school days and weeks.

The best response, she argues, based on years of research, is to reduce class size and give students the attention and care they need to make up for lost time.

Categories Accountability, Class size, Education Reform, NAEP
« Older Entries