Archives for category: NAEP
« Older Entries

Paul Thomas: Big Lies of Education: Confusing NAEP Proficient with “Grade Level”

By dianeravitch
February 28, 2024 //
25

I served on the National Assessment Governing Board from 1998-2004. NAGB is the governing agency for NAEP, the federal test. I was appointed by President Bill Clinton. I learned about the inner workings of standardized testing, much of which made me skeptical of it.

I have often observed that critics of public schools assume that NAEP Proficient is the same as “grade level,” when in fact NAEP warns readers explicitly in every score report that NAEP Proficient is NOT “grade level.” In fact, NAEP Proficient represents mastery of what was tested, which I would characterize as an A or A-.

In 2010, when the anti-public school documentary “Waiting for ‘Superman’” was released, I reviewed it for The New York Review of Books and criticized it for confusing NAEP Proficiency with grade level, then claiming that most American kids can’t read, all because of their terrible public schools, their terrible teachers and those awful unions. The way to a better future, the documentary claimed, was charter schools. Not true. Even Secretary of Education Miguel Cardona has repeated this erroneous claim. Apparently neither he nor his speech writers reads NAEP reports with care and no one has briefed them.

I have explained this confusion on several occasions on the blog. I even called the Commissioner of the National Center on Education Statistics and proposed that NAEP Proficient be renamed “NAEP Mastery,” to clarify its meaning. She sounded enthusiastic about the idea (which came from a reader of this blog) but nothing changed.

I am very happy to see that Professor Paul Thomas at Furman University in South Carolina has launched a series called “Big lies in Education,” and this claim is one of the Big Lies. It is a lie because the fact that NAEP Proficient is not grade level is stated plainly in every release of NAEP scores.

Thomas begins:

One of the most bearish statistics for the future of the United States is this: Two-thirds of fourth graders in the United States are not proficient in reading,” wrote Nicholas Kristof in The New York Times.

Kristof’s piece in 2023 can be traced back to a similar claim by Emily Hanford in 2018: “More than 60 percent of American fourth-graders are not proficient readers, according to the National Assessment of Educational Progress, and it’s been that way since testing began in the 1990s,” including a surprisingly ineffective graphic.

Open the link to see this and other graphics.

The student reading proficiency Big Lie grounded in misrepresenting or misunderstanding NAEP is likely one of the most complicated Big Lies of Education.

In media and political rhetoric, first, the terms “reading proficiency” and “grade level reading” are commonly jumbled and used inappropriately as synonyms.

Achievement levels such as “basic” and “proficient,” such as used in NAEP for reading, are misleading and complicated for most people not familiar with technical terminology.

NAEP “basic” is approximately grade level (although even that claim is problematic since no standard exists in the US for “proficient” or “grade level”), and “proficient” on NAEP is high: 

Another important graph. Open the link.

Hanford’s and Kristof’s Big Lie, then, is a combination of blurring NAEP achievement levels with grade level reading achievement and manufacturing a reading crisis with that misinformation.

Ironically, NAEP grade 4 reading scores for a decade show that 2/3 of students are reading at or above grade level, the inverse of the false crisis claims of the media:

Open the link for the graph.

This is an excellent expose, which everyone should read. The claim that most kids read below grade level is foundational to the claim that public schools are in crisis. Its a Big Lie.

Categories Accountability, Corporate Reform, Fake, Lies, Literacy, NAEP, Standardized Testing

What Do Test Scores Measure? How I Lost My Faith in Their Value

By dianeravitch
February 14, 2024 //
58

For many years, I was a staunch advocate of standardized testing. But I lost my enthusiasm for standardized testing after spending seven years on the governing board of NAEP (the National Assessment of Educational Progress). NAEP is the federal test administered every two years to measure academic progress in reading and math, as well as testing other subjects. The test takers are randomly selected; not every student answers the questions on any test. There are no stakes attached to NAEP scores for any student, teacher, or school. The scores are reported nationally and by state and for nearly two dozen urban districts. NAEP is useful for gauging trends.

Why did I lose faith in the value of standardized testing?

First, over the course of my term, I saw questions that had more than one right answer. A thoughtful student might easily select the “wrong” answer. I also saw questions where the “right” answer was wrong.

Second, it troubled me that test scores were so highly correlated with socioeconomic status. Invariably, the students from families with the highest income had the highest scores. Those from the poorest families had the lowest scores.

Third, the latter observation spurred me to look at this correlation between family wealth and test scores. I saw it on the results of every standardized test, be it the SAT, the ACT, or international tests. I wondered why we were spending so much money to tell us what we already knew: rich kids have better medical care, fewer absences, better nutrition, more secure and stable housing, and are less likely to be exposed to vermin, violence, and other health hazards.

Fourth, when I read books like Daniel Koretz’s “Measuring Up” and “The Testing Charade” and Todd Farley’s “Making the Grades: My Misadventures in the Standardized Testing Industry,” my faith in the tests dissipated to the vanishing point.

Fifth, when I realized that the results of the tests are not available until the late summer or fall when the student has a new teacher, and that the tests offer no diagnostic information because the questions and answers are top-secret, I concluded that the tests had no value. They were akin to a medical test whose result is available four months after you see the doctor, and whose result is a rating comparing you to others but utterly lacking in diagnostic information about what needs medication.

So, all of this is background to presenting a recent study that you might find useful in assessing the value of standardized tests:

Jamil Maroun and Christopher Tienken have written a paper that will help you understand why standardized tested is fatally flawed. The paper is on the web and its title is:

The Pernicious Predictability of State-Mandated Tests of Academic Achievement in the United States

Here is the abstract:

The purpose of this study was to determine the predictiveness of community and family demographic variables related to the development of student academic background knowledge on the percentage of students who pass a state-mandated, commercially prepared, standardized Algebra 1 test in the state of New Jersey, USA. This explanatory, cross-sectional study utilized quantitative methods through hierarchical regression analysis. The results suggest that family demographic variables found in the United States Census data related to the development of student academic background knowledge predicted 75 percent of schools in which students achieved a passing score on a state standardized high school assessment of Algebra 1. We can conclude that construct-irrelevant variance, influenced in part by student background knowledge, can be used to predict standardized test results. The results call into question the use of standardized tests as tools for policy makers and educational leaders to accurately judge student learning or school quality.

The paper was peer-reviewed. It was published last week.

Categories Accountability, Education Industry, Education Reform, NAEP, Standardized Testing

Paul Bonner: Our Useless Focus on Test Scores

By dianeravitch
December 19, 2023 //
68

For the past quarter-century, American policymakers have been laser-focused on raising test scores. They assumed that higher test scores equals better education equals better economy. The cost of all this testing was billions of dollars, which would have been better spent on reducing class sizes, raising teachers’ salaries, and updating schools.

From No Child Left Behind to Race to the Top to the Every Student Succeeds Act of 2015, federal policy has made standardized test scores the most consequential measure of all schooling. Every release of scores by the National Assessment of Educational Progress produces a flurry of articles with dire predictions about the future (“a Sputnik Moment!) or the discovery of a miracle (e.g. the Texas/Florida/Mississippi 4th grade reading miracle, which strangely disappears by the 8th grade).

But an occasional outbreak of wisdom cautions us that we are looking for “success” in the wrong place.

Paul Bonner is a retired educator. He posted the following comment on the blog.

My first personal encounter with NAEP was around 2005. I was an eighth grade assistant principal facilitating the process between my staff and the NAEP testing officials who were to give the test. As I monitored the hall during the testing of selected random students, it struck me how disinterested our students were in performing on the assessment. My school at the time was a high performing magnet program with a highly motivated student body. I assumed, incorrectly, that due to the competitive attitudes of our students that they would want to perform well, as I had with standardized assessments in the 1970s no matter what it meant concerning my academic standing. What I learned in this first encounter was that students were already fed up with standardized tests particularly if it had no bearing on their academic standing. These students made a habit of blowing away all of the state tests and for them NAEP was a waste of time. The idea of NAEP as a report card might be significant if students were not already wasting three weeks of their year with state and district tests. In other words, no student benefit so why bother. How does this give us an accurate read on student capacity? Second, none of the standardized assessments, international, national, state, or local have shown meaningful movement in student performance over the decades. A few points either way does not reveal any real change in instructional efficacy or evidence of greater learning opportunities for students no matter their circumstances. The realities remain the same. Students prepared for schooling or provided significant instructional and experiential resources perform well. Those who do not have such privilege do not. Policy makers and educational leaders are simply fooling themselves when denying that fact. Maslow’s Hierarchy of Needs is telling in this regard. Piddling about a few point improvement in a NAEP test for fourth graders isn’t going to change the fact that Mississippi and other poor states provide far less opportunity for their students and poorer outcomes than wealthier states wiling to put more resources in the classroom. Testing has become a waste of time and money that could be better used elsewhere.

Categories Accountability, Competition, Every Student Succeeds Act (ESSA), Hoax, NAEP, NCLB (No Child Left Behind), Race to the Top, Standardized Testing

North Carolina: Federal Education Official Involved in Charter School’s Financial Woes

By dianeravitch
December 14, 2023 //
13

Dr. Peggy Carr is Commissioner of the National Center on Education Statistics, a prestigious, major federal agency. NCES preceded the U.S. Department of Education by more than a century, having been created by Congress in 1867 to report on the progress and condition of American education. NCES releases regular reports on education. It also oversees the National Assessment of Educational Progress (NAEP), the federal testing agency.

T. Keung Hui of the Charlotte Observer reported that Dr. Carr is ensnared in a state investigation of a charter school called Children’s Village Academy and its financial affairs. The school’s charter is up for renewal in 2024.

A North Carolina charter school is being accused of misspending thousands of taxpayer dollars, including funds spent on behalf of a high-ranking federal education official who is a leader at the school.

Staff from the state Department of Public Instruction this week presented reports alleging conflict of interest violations involving the spending of state and federal dollars at Children’s Village Academy in Kinston. Many of the questions revolved around money exchanged between the school and its board vice chair Peggy Carr, who is also commissioner of the National Center for Education Statistics.

Specific concerns include Carr getting $155,000 in interest payments on a $188,000 loan she gave the school 15 years ago. Other allegations include the school improperly using taxpayer dollars to reimburse Carr for furniture and utility bills for a home she owns and rents to the school in the summer….

In 2008, Carr gave the school a $188,000 loan that is still being repaid. DPI says there was inadequate documentation of the loan , resulting in misstatement of the school’s finances because it wasn’t listed as being a liability..

McFadden said that Carr has been paid back, with interest, $314,000. But by the time the loan is fully repaid, McFadden said the school will have paid an estimated $155,505 in interest — $109,268 more than it was originally projected to repay.

“DPI is concerned with the legality and validity of the loan payments to date since there is no documentation or evidence that substantiates the CVA Board agreed to or understood the total amount to be paid including interest based on the annual decisions being made,” according to a DPI report.

In addition, DPI has questions about the $894 a month it says Children’s Village is paying to reimburse Carr for small business loans for buildings the school uses…

DPI identified $5,003 in “unallowable costs,” from the summer program, including $4,438 for furnishings that Carr purchased and requested reimbursement for at a house she partially owns in Kinston.

The school leases the home for two months a year for the summer program, DPI says. Items purchased included dining room tables, dining room chairs and decorative items such as a wall mirror, “colorful cows” and pillows. Some of the items were purchased in Maryland, where Carr lives, and shipped to Kinston.

“Per contracts for the property where the furnishings are used, the property is only used for 2 months out of the year,” according to a DPI report. “The furnishings in question are also not a reasonable purchase as they are typically found in a household, they are not furnishings typically found in an academic setting.”

In addition, DPI says the school paid the entire utility bill for the house for two summer months even though part of the property was used by an independent contractor who is related to Carr. That person is the school’s operations manager. A U-Haul business is also in that building.

Even after the summer program ended, DPI says the school paid the utility bills for the home. Altogether, DPI found $3,238 in unallowable utility costs that must be repaid….

DPI outlined a list of other questioned costs, including:

▪ A custodian was paid $17,000 in federal summer program grant month for July through September.

▪ A different custodian/bus driver who is married to the K-5 principal was paid $15,000 in federal grant dollars in July and August. The K-5 principal is also Carr’s sister.

▪ DPI found $8,877 in unallowable costs related to personal expenditures such as a tire replacement for the finance officer’s car, holiday gifts to employees, $500 gift cards to four employees and costs related to a daycare center operating on the campus. McFadden said the daycare owner is related to Carr.

Read more at: https://www.newsobserver.com/news/local/education/article282963048.html#storylink=cpy

Categories Accountability, Charter Schools, Data, Education Industry, NAEP, North Carolina, Research, Scandal, U.S. Department of Education

Massachusetts, Top State on NAEP, Discovers a “Serious Literacy Crisis” Among Youngest Children

By dianeravitch
December 4, 2023 //
28

This is the most bizarre story I have read in many a day. The Boston Globe reported on a study showing a “serious literacy crisis” among the state’s youngest children. This is strange because Massachusetts regularly performs at the top of NAEP reading assessments.

The study was conducted by WestEd, a research group based in California. Apparently the researchers assessed the literacy skills of children in kindergarten, first and second grades. It is not surprising that most children in K and 1 and even 2 can’t read. They are only beginning to read.

The story starts:

A new state-commissioned study of young elementary students found that more than half showed early signs of reading difficulties — more evidence that the state has a serious literacy crisis, despite its reputation for educational excellence.

The report, released Friday, provides a first-of-its kind look at the reading skills of the state’s youngest children, whose reading prowess is not assessed by the state until the first MCAS exam in third grade.

The results are troubling: Nearly 30 percent of students in grades K-3 were at high risk of reading failure, and as many as 20 percent showed signs of having dyslexia, a language processing disorder that must be addressed with specialized reading instruction. Low-income students, those learning English or receiving special education services, Latino students, and Black students were most likely to experience reading struggles, according to researchers with WestEd, a San Francisco-based nonprofit that conducted the analysis.

The report suggests schools are not helping most struggling readers catch up: 60 percent of students who began the school year at risk of reading difficulties ended the school year in the same concerning position. But it found that younger students are much more likely to improve with extra help than older students are, a powerful argument for early intervention…

The extent of the state’s early literacy struggles have been laid bare annually in MCAS results, which, as the Globe’s Great Divide team previously reported, regularly show tens of thousands of students advancing from grade to grade without the reading skills they need to be successful.

The Globe investigation found nearly half of the state’s school districts last school year were using a reading curriculum the state considered “low quality.” A national nonprofit ranked Massachusetts this year in the bottom half of the nation in preparing educators to teach reading.

Massachusetts has not, as other states have, required evidence-based methods of reading instruction.

The “national nonprofit” that gave low scores to teacher education programs in the state is the National Council on Teacher Quality, a conservative group created by the Thomas B. Fordham Foundation and the George W. Bush administration. Its goal is to promote phonics. When NCTQ ranks Ed schools, it doesn’t visit them; it reads their catalogues.

If Massachusetts has a “serious literacy crisis,” the rest of the nation is a dumpster fire.

On NAEP, fourth grade students in Massachusetts typically score at or near the top in the nation. The percentage of students in Massachusetts who performed at or above NAEP Proficient in 2022 was 43%.

NAEP Proficient is equivalent to an A.

The only jurisdiction with higher scores in fourth grade was the Department of Defense schools. Five states had scores that were not significantly different from Massachusetts. Those six states outperformed 45 states and jurisdictions in fourth grade.

The point of the WestEd study seems to be that the state must push through a greater emphasis on phonics in teacher education programs, and that MCAS testing in grade 3 should start sooner.

The children who need extra help are low-income, limited-English, or in need of special services, etc. This is not news.

The “serious literacy crisis” looks and smells like a manufactured crisis. This report looks like a hit job on the state’s teachers and colleges of education. If the rest of the nation’s children matched the performance of those in Massachusetts, that would be cause for a national celebration.

Categories Childhood, Education Reform, Literacy, Massachusetts, NAEP, Standardized Testing

Jeb Bush: Don’t Abandon High-Stakes Testing (Except for Voucher Schools)

By dianeravitch
December 2, 2023 //
8

Jeb Bush, a founding father of the corporate reform movement, was governor of Florida from 1999 to 2007. He implemented a regime of high-stakes standardized testing, third grade retention, school report cards, and choice. He vigorously championed charter schools and tried to change the state constitution to allow vouchers for religious schools. Now he is concerned that the legislature might undermine high-stakes testing, so he penned this opinion piece for the Orlando Sentinel bragging about the success of his test-and-punish regime.

Yes, Florida’s fourth-grade NAEP scores are high. But he does not acknowledge that the scores are high because Florida “retains” third-graders who don’t pass the reading test. Holding these kids back artificially inflated the fourth-grade scores. By eighth grade, Florida’s scores are at the national average. Nothing to boast about there. The moral of the story: retention raises test scores by removing from the testing pool the kids who were retained (flunked).

The other curious omission in this article is voucher schools. Jeb is a huge fan of vouchers but voucher schools don’t take any state tests. How does he explain this? He doesn’t.

He wrote:

For more than two decades, Florida has remained committed to educational excellence by ensuring that transparency, accountability and opportunity define our K-12 system. We’ve consistently pushed the envelope, transforming Florida into a national leader. This has not happened by accident.

When I took office, nearly half of Florida’s fourth graders had significant reading deficiencies. Similarly, half of Florida’s fourth graders were significantly below grade level in math. Only half of high school students graduated on time.

In partnership with state lawmakers, we championed the A+ Plan in 1999 based on core principles of high expectations, standardized measurement, a clear and achievable system of accountability, rewards and consequences for performance, effective teaching in the classroom and more choices for families to customize an education for each student.

Today, Florida’s fourth graders rank third in the nation for reading achievement and fourth in the nation for math achievement. Our high school graduation rate is approaching 90%.

This is why it’s concerning that some lawmakers now seem eager to throw out or water down key components of the policies that led our students from the back of the pack to top in the nation.

I understand the goal of the Florida Senate’s recently unveiled deregulation package (Senate bills 7000, 7002and 7004). Cutting red tape and removing outdated regulations is a worthwhile effort.

But this cannot come at the cost of our state and students taking a step backward.

Lawmakers have proposed watering down our third grade literacy policy, removing the backstop of retention and paving the way to reinstate social promotion. Requiring that students objectively demonstrate they are reading successfully before being promoted to fourth grade has been a core part of Florida’s comprehensive early literacy policy — one that research has consistently supported.

This is why it’s concerning that some lawmakers now seem eager to throw out or water down key components of the policies that led our students from the back of the pack to top in the nation.

I understand the goal of the Florida Senate’s recently unveiled deregulation package (Senate bills 7000, 7002and 7004). Cutting red tape and removing outdated regulations is a worthwhile effort.

But this cannot come at the cost of our state and students taking a step backward.

Lawmakers have proposed watering down our third grade literacy policy, removing the backstop of retention and paving the way to reinstate social promotion. Requiring that students objectively demonstrate they are reading successfully before being promoted to fourth grade has been a core part of Florida’s comprehensive early literacy policy — one that research has consistently supported.

Most parents believe their children are reading on grade level even when they are not. Florida’s retention policy raises expectations. We know there are grave later-life outcomes for struggling readers. Lowering expectations by watering down the retention requirement will not help students in third grade or beyond.

Moreover, abandoning the requirement that Florida students pass the tenth grade English Language Arts and Algebra I end-of-course assessments further reduces expectations and hampers Florida’s workforce development efforts. Removing this requirement may aid Florida’s graduation rates, but it will reduce the diploma to nothing more than a participation certificate.

If we expect less, we will get less. This cannot be the future we want for Florida.

Finally, part of the package would turn back recent gains for charter schools to be treated equitably alongside their traditional public school peers. The bill’s proposed changes would make it harder for charter schools to access vacant public school buildings and reduce the share of Title I funds made available to students attending charter schools. This is a step backward.

Maintaining Florida’s system of high expectations, clear accountability and robust choice is as important to our future as anything. We’ve spent two decades establishing, maintaining and building upon these ideals.

Now is not the time for lawmakers to get weak-kneed on policies that have played key roles in contributing to two decades of educational progress.

Jeb Bush was governor of Florida from 1999 to 2007. He is the founder and chair of ExcelinEd, ExcelinEd in Action and the Foundation for Florida’s Future.

Categories Accountability, Bush, Jeb, Charter Schools, Corporate Reform, Education Industry, Extremism, Failure, Literacy, NAEP, Standardized Testing, Vouchers

John Thompson: What Do School Report Cards Mean?

By dianeravitch
November 28, 2023 //
7

John Thompson, historian and retired history teacher, analyzes the use and misuse of Oklahoma’s school report cards.

He writes:

As usual, the 2023 Oklahoma school Report Card prompted headlines about “struggling” students. But counter-intuitively, State Superintendent Ryan Walters stressed the declines during his time in office!?!?

Two tales of the Report Card are being told. As the Tulsa World reports, Walters “claimed that the data was from ‘previous years,’ even though all of the academic achievement indicators are from state tests administered just seven months ago.” Yes, taking office as State Superintendent in January 2023, Walters hasn’t had time to achieve many gains in learning, even if he’d really tried to. But the chaos during 2023, combined with the disruption he’d spread since 2020 as head of the Education Department, provided plenty of time for disruption.

As the Oklahoman reports, Walters cited the greatest decline under his watch, 8th grade reading proficiency which saw “a 5.7 percent decrease,” although “No other grade had more than a 0.4 percent decrease in reading scores, and some others “saw a very small uptick in reading scores.” Walters then promised “we are taking a Back to Basics approach,” which is the opposite of what it takes to increase proficiency.

The wisest narrative, illustrated by the Education Watch’s Jennifer Palmer, places the 2023 Report Card within the context of the massive decline of scores due to Covid, and the 2022 report. The 2023 report saw “no big swings in proficiency rates in any of the three tested subjects content,” while noting the overlooked fact that “a score of basic means a student demonstrated foundational knowledge and skills.”

Then Palmer tweeted background information on the differences between what basic means, as opposed to the widely misunderstood grade of proficiency which, I must add, has been misrepresented since the Reagan administration in order to denigrate public education. Oklahoma’s 8th grade reading proficiency grade requires that “students demonstrate mastery over even the most challenging grade-level content and are ready for the next grade, course or level of education.” It requires mastery of grade level skills that include interpretation, evaluation, analysis across multiple texts, and critical thinking. Mastery in requires use of evidence, argumentative response and synthesis of to create “written works for multiple purposes.”

As Palmer tweeted, we need a more nuanced” understanding of “reading.” And “the 8th graders who didn’t score proficient, but are in the ‘basic’ category, can still do all this” and then she linked to the challenging goals that are required for that grade, which include: partial mastery of interpretation, evaluation, analysis across multiple texts, critical thinking, use of evidence, argumentative response and synthesis.

Granted, these definitions are not necessarily the same as the more reliable NAEP scores. But as Jan Resseger explains, the nation’s NAEP proficiency grade “represents A level work, at worst an A-” and, basically, the same applies to Oklahoma’s tests. She asks, “Would you be upset to learn that “only” 40% of 8th graders are at an A level in math and “only” 1/3rd scored an A in reading?”

Ressenger also cites the huge body of research explaining why School Report Cards aren’t a reliable tool for measuring school effectiveness. We need a better understanding why the proficiency has been weaponized against schools, but we also need to master the huge body of research which explains why Report Cards aren’t a fair, reliable, and valid measure of how well schools are performing.

I’ll just cite one of the scholars that Ressenger draws upon. Stanford’s Sean Reardon’s 2022 research explained why “test score gaps may result from unequal opportunities either in or out of school; [but] they are not necessarily the result of differences in school quality, resources, or experience.” Reardon documented:

The socioeconomic profile of a district is a powerful predictor of the average test score performance of students in that district. The most and least socioeconomically advantaged districts have average performance levels more than four grade levels apart. … Achievement gaps are larger in districts where black and Hispanic students attend higher poverty schools than their white peers… and where large racial/ethnic gaps exist in parents’ educational attainment. The size of the gaps has little or no association with average class size, a district’s per capita student spending or charter school enrollment.

And that brings us to chronic absenteeism. As the New York Times reports, across the nation, “nearly 70 percent of the highest poverty schools experienced widespread, chronic absenteeism in the 2021-22 school year,” and “in these schools, about a third or more of the student body was considered chronically absent.” Of course, the Times notes, “Students cannot learn if they are not in school, and they cannot benefit from interventions, such as tutoring, that are supposed to help them make up pandemic losses.”

And Palmer reports:

Across the state, 20% of students were chronically absent last year, a half a percent increase over 2022. Some student groups were even higher: 24% of Hispanic students, 25% of economically disadvantaged and 31% of Black students were chronically absent …” Moreover, excessive absences are more prevalent now than before the pandemic. In 2019, 14% of Oklahoma students were chronically absent.

Tulsa World had previously reported that “About half of the Tulsa high school students are chronically absent” and explained why this complex and serious problem is “showing no signs of improvement.” The World cited the work of Georgetown’s Phyllis Jordan who explained the need to reconnect “what’s going on in the school and what’s going on outside the school.”

On one hand, that is why Patrick Forsyth, a University of Oklahoma professor who had analyzed the state’s A-F report card system, said “using attendance to measure school effectiveness is like using rates of tobacco use to measure hospital effectiveness.” On the other hand, as the Oklahoman reported, the Attendance Works’ Hedy Chang said, chronic absenteeism is an “all-hands-on-deck moment.” She also called on schools to “learn the specific barriers to attendance that their students experience before crafting a response to those unique challenges.”

That gets us back to the tragedy of two tales about what the Report Card means. Sadly, Ryan Walters uses it as one more weapon for disrupting public education. The other side must use these flawed metrics not to punish but for diagnostic purposes.

Categories Education Reform, NAEP, Oklahoma, Standardized Testing

Paul Thomas: Our Perpetual “Crisis in Education” Is Fueled by Unreliable, Invalid Test Scores

By dianeravitch
November 3, 2023 //
10

Paul Thomas of Furman University is a clear-sighted analyst of education policy. He is fearless when it comes to calling out frauds. This post is a good example.

He writes:

“The administrations in charge,” write Gilles Deleuze in Postscript on the Societies of Control, “never cease announcing supposedly necessary reforms: to reform schools, to reform industries, hospitals, the armed forces, prisons” (p. 4).

Deleuze’s generalization about “supposedly necessary reforms” serves as an important entry point into the perpetual education crisis in the US. Since A Nation at Risk, public education has experienced several cycles of crisis that fuel ever-new and ever-different sets of standards and high-stakes testing.

Even more disturbing is that for at least a century, “the administrations in charge” have shouted that US children cannot read—with the current reading crisis also including the gobsmacking additional crisis that teachers of reading do not know how to teach reading.

The gasoline that is routinely tossed on the perpetual fire of education crisis is test scores—state accountability tests, NAEP, SAT, ACT, etc.

While all that test data itself may or may not be valuable information for both how well students are learning and how to better serve those students through reform, ultimately all that testing has almost nothing to do with either of those goals; in fact, test data in the US are primarily fuel for that perpetual state of crisis.

Here is the most recent example—2023 ACT scores:

I have noted that reactions and overreactions to NAEP in recent years follow a similar set of problems found in reactions/overreactions to the SAT for many decades; the lessons from those reactions include:

  • Lesson: Populations being tested impact data drawn from tests.
  • Lesson: Ranking by test data must account for population differences among students tested.
  • Lesson: Conclusions drawn from test data must acknowledge purpose of test being used (see Gerald Bracey).

The social media and traditional media responses to 2023 ACT data expose a few more concerns about media, public, and political misunderstanding of test data as well as how “the administrations in charge” depend on manipulating test data to insure the perpetual education crisis.

Many people have confronted the distorting ways in which the ACT data are being displayed; certainly the mainstream graph from Axios above suggests “crisis”; however, by simply modifying the X/Y axes, that same data appear at least less dramatic and possibly not even significant if the issues I list above are carefully considered….

This crisis-of-the-day about the ACT parallels the central problem with NAEP, a test that seems designed to mislead and not inform since NAEP’s “Proficient” feeds a false narrative that a majority of students are not on grade level as readers.

The ACT crisis graph being pushed by mainstream media is less a marker of declining educational quality in the US and more further proof that “the administrations in charge” want and need testing data to justify “supposedly necessary reforms,” testing as gas for the perpetual education crisis fire.

Please open the link to read this excellent analysis in full.

Categories Accountability, Corporate Reform, Disruption, Education Industry, Education Reform, Failure, Hoax, Lies, Literacy, NAEP, Standardized Testing

Arnold Hillman: How to Reform NAEP Scores: A Satire

By dianeravitch
July 15, 2023 //
18

Arnold Hillman is a retired educator who spent his career in Pennsylvania and retired in South Carolina. Bear this in mind when you read his satire. Must be the SC water.

The decline of both reading and math scores on the NAEP national test is a harbinger of a predictive outpouring of solutions to the problem. That has been the standard for the last 100 years of public education. We typically find panaceas to “fix” problems in education.

Here is a very simple one. Until the beginning of the 20th century, education was rather simple- teach reading, writing and arithmetic. On the side you might provide vocational programs. However World War I provided us with a look into the future.

Many of the conscripts in the American army were seen not to be physically fit. That was a danger in a war. There was no part of the constitution that mentions education. The idea of a healthy mind and healthy body was promulgated by none other than John Dewey. World War I was an instigator, and schools took up the mantle.

That’s how things change in education. The nation needed more scientists to combat Russia’s preeminence in space and so Congress passed the National Defense Education Act (NDEA). I know that you are getting the idea now. If you live long enough, you will see even more of these things.

Now, how will the decline in these scores be cured by those with the money to do it. Seems like administrations these days are not in the business of fixing education. You can tell by all ofthe news about investigations, indictments, Russian problems and all sort of other adjuncts to those happenings. So then, who or what will come through to help us climb out of this educational abyss?

Lets try this on for size. How about the Broad Foundation. Let’s give them leave to train all of the school superintendents in the nation. That’s only 13,452 school district superintendents. With all of the resources available to the foundation, this could be accomplished in the wink of an eye (see the movie “I Robot” for a reference).All problems of reading and math will follow the same successes that the Broadies have had in all of the places where they have been installed as superintendents. That’s for sure.

Let then have the voucher folks come up with the plan to take over public schools and do their level best to cherry pick the students that they will help. There will certainly be some unintended consequences, such as massive dropouts, higher crime rates, more unemployment and many other charming things.

These voucher folks have a way with statistics. In their first year of operation, math and reading scores will soar. All students will be on grade level in reading and all of them will be up to fractal geometry, after surpassing the highest scores ever on the NAEP test.

Another challenger will be the charter school folks. All schools could be “charterized” and escape from the silly laws that restrict public schools in their education of kids. Since charters do not have to have all certified teachers, that will be a great advantage. We can then dismiss those pesky teachers who have not been doing a good job anyway.

There would not be any responsibility for those charters to have any parental involvement. Parents or guardians will only know what is going on when their child gets a report card.

Huge management companies will continue to “buy up” these charters and run them for profit. The movement to make these charters non-public has already happened in the Washington state Supreme Court. It has decided that Charter Schools were not, in fact, public schools.

Think of all the improvements that charter schools have made across the country since their inception in Minnesota. We can have a myriad of online charter schools which will definitely improve reading and math scores, especially in kindergarten.

We are fortunate to have a parents group that is very interested in improving education by going onto the nation’s school boards and making things so much better when they are there. Incompetent administrators are fired by the dozens and reading and math scores have already risen as a result of these actions.

The premiere group is called “ Moms for Liberty.” Not sure why there are no Dads included. There must be a Title IX reason. These folks have the kind of enviable clout that gets these students on their way to improving their math and reading scores.

With “Moms for Liberty” in charge, schools will have the advantage of being close to those who lead our country. They are proud to have national figures, some even running for President, who will make sure the schools are doing the right thing.

Then we have a group that includes some very wealthy folks. Some of them are anonymously giving funding and directions to those who were described earlier. They are famously supporters of vouchers, privatization of public schools, charters and the like. They support parent groups like “Moms for Liberty.” Their aims are certainly to help students improve their reading and math scores. We will call them, for better or worse, “ The Billionaire Class.”

With all of these folks helping out, how long do you believe it will take for our youngster’s math and reading scores to soar?

Categories Billionaires, Broad Foundation, Corporate Reform, Education Reform, Humor, Innovation, NAEP, Standardized Testing

Gene V. Glass: The Sky Truly Is Falling, Somewhere

By dianeravitch
July 13, 2023 //
35

Perhaps you saw the story in the New York Times a few days ago, lamenting that American students were not making up the ground they lost academically during the pandemic. This was presented as a full-blown crisis. The period from March 2020 to the fall of 2022 included many disruptions: family members died or were very sick, teachers and other school staff died or were very sick, many schools closed, many adopted online classes, normal life came to an end for more than two years, affecting family life and mental health.

When I read the panicked discussion in the New York Times, based on a study by NWEA, a major standardized testing company, I reached out to one of the wisest people I know and asked him to discuss the issues. That’s Gene V. Glass, one of the nation’s eminent education researchers. He wrote the following commentary for the blog.

He wrote:

The New York Times is worried — no, it’s panic-stricken.

Jeremiah — the weeping prophet of the Torah — was a veritable Pollyanna compared to today’s policy wonks and political animals when it comes to delivering bad news.

Journalists — like the New York Times education reporter Sarah Mervosh — envision the half-dozen NAEP score point loss from 2019 to 2022 to result in fewer students accepted to college with a resulting lifetime loss of $70,000 income. Does anyone seriously entertain the possibility that colleges will forego the tuition & fees payments of students who scored 5 points lower in the eighth grade on the NAEP test? Nor did anyone weep in 1990 when the scores were 30 points lower and lifetime incomes were doing just fine, thank you?

Three weeks ago, Dana Goldstein reported, again in the New York Times, that on the October 2022 administration of the NAEP test, the 13-year-olds scored an average of 256 in reading – down 4 pints — and 271 in math — down 9 points — from the averages in 2019. What happened between 2019 and 2022? The COVID pandemic, of course.

Conjuring up the causes of NAEP fluctuations has grown into a widespread academic game since the NAEP Governing Board decided to call some scores “Advanced” and other scores “Basic” — euphemisms for “Excellent!” and “My, my; we have a problem here.” When the U.S. History NAEP scores dropped 9 points from 2014 to 2022, no one panicked, or even cared apparently. The Civics NAEP score never wavered more than 1 or 2 points between 1998 and 2022; the average was 150 in 1998 and 150 again in 2022, never rising above 153. Again, a big nothing, though arguably the nation needs better citizens than it needs better calculators.

NAEP scores jump around for all sorts of reasons, mostly a host of very small reasons that are impossible to unravel. Readers can exercise their own analytic muscles puzzling over NAEP scores. Reading scores were already declining from 2017 to 2019 before the pandemic. In 2019, NAEP Mathematics scores increased at grade 4, but decreased at grade 8 compared to 2017. Moreover, the percent of 13-year-olds taking algebra declined from 34 percent in 2012 to 24 percent today. I don’t disapprove of such a decline; I merely advance it as one of a number of reasons why NAEP scores might be wiggling from time to time. And even bigger forces are afoot.

A couple dozen Red states are experiencing a mass migration of middle-class and upper-class white students to private and charter schools, all driven with barely a whimper by crypto-privatization backers. It would be remarkable for NAEP scores to hold steady under these circumstances alone. Arizona’s universal school voucher program is expected to cost the taxpayers $900 million for the 2023-24 year, more than ten times initial estimates. Three-quarters of the initial voucher applicants are students already in charter, private, and parochial schools. The U.S. percentage of white K-12 students in public schools is projected to decline by 20% between 2010 and 2030.

But the occasion of the COVID pandemic has become a good excuse for policy analysts to propose their favorite solution to the crisis, indeed a “manufactured crisis” in the words of my friend and colleague David Berliner who introduced the term in 1995. Experts have offered no fewer than a dozen emergency measures needed for the nation’s recovery. Nine of these follow: 1. Smaller classes; 2. Tutoring; 3. Extending the school year; 4. Adding a fifth year of high school; 5. Focused funding on minority students; 6. Focusing on math, not reading; 7. Full-time summer school; 8. Increased teacher pay; 9. Focusing on students’ mental health. A fifth year of high school?! Seriously? Now I’m in favor of most of these things, except focusing on math. How about Civics?

Some seem to favor no solution at all. A leitmotif of the NYT article is that the federal government’s billions of dollars in pandemic emergency aid to schools were misspent. The trouble is, less will be heard of any of these “solutions” after NAEP scores wiggle up a couple of points in the next testing.

The ravages of the pandemic were caused by a White House of Dunces. Incompetence cost the nation 1.1 million lives, two or three times more than the irreducible number. It will never happen again — at least not with COVID. The stewardship of all children’s education is a responsibility of every citizen, regardless of the next year’s tweak in NAEP scores. Put that on NAEP’s next Civics test.

Gene V Glass

Emeritus Regents’ Professor

Arizona State University

 

The New York Times is worried — no, it’s panic-stricken. https://shorturl.at/mtI15

 

Jeremiah — the weeping prophet of the Torah — was a veritable Pollyanna compared to today’s policy wonks and political animals when it comes to delivering bad news. 

 

Journalists — like the New York Times education reporter Sarah Mervosh — envision the half-dozen NAEP score point loss from 2019 to 2022 to result in fewer students accepted to college with a resulting lifetime loss of $70,000 income. Does anyone seriously entertain the possibility that colleges will forego the tuition & fees payments of students who scored 5 points lower in the eighth grade on the NAEP test? Nor did anyone weep in 1990 when the scores were 30 points lower and lifetime incomes were doing just fine, thank you? 

 

Three weeks ago, Dana Goldstein reported, again in the New York Times, that on the October 2022 administration of the NAEP test,  the 13-year-olds scored an average of 256 in reading – down 4 pints —  and 271 in math — down 9 points — from the averages in 2019. https://shorturl.at/DFHZ5  What happened between 2019 and 2022? The COVID pandemic, of course. 

 

Conjuring up the causes of NAEP fluctuations has grown into a widespread academic game since the NAEP Governing Board decided to call some scores “Advanced” and other scores “Basic”  — euphemisms for “Excellent!” and “My, my; we have a problem here.”  When the U.S. History NAEP scores dropped 9 points from 2014 to 2022, no one panicked, or even cared apparently. And the Civics NAEP score never wavered more than 1 or 2 points between 1998 and 2022; the average was 150 in 1998 and 150 again in 2022, never rising above 153. Again, a big nothing, though arguably the nation needs better citizens than it needs better calculators.  

 

NAEP scores jump around for all sorts of reasons, mostly a host of very small reasons that are impossible to unravel. Readers can exercise their own analytic muscles puzzling over NAEP scores at www.nationsreportcard.gov. Reading scores were already declining from 2017 to 2019 before the pandemic. In 2019, NAEP Mathematics scores increased at grade 4, but decreased at grade 8 compared to 2017. Moreover, the percent of 13-year-olds taking algebra declined from 34 percent in 2012 to 24 percent today. I don’t disapprove of such a decline; I merely advance it as one of a number of reasons why NAEP scores might be wiggling from time to time. And even bigger forces are afoot. 

 

A couple dozen Red states are experiencing a mass migration of middle-class and upper-class white students to private and charter schools, all driven with barely a whimper by crypto-privatization backers. It would be remarkable for NAEP scores to hold steady under these circumstances alone. Arizona’s universal school voucher program is expected to cost the taxpayers $900 million for the 2023-24 year, more than ten times initial estimates. Three-quarters of the initial voucher applicants are students already in charter, private, and parochial schools. The U.S. percentage of white K-12 students in public schools is projected to decline by 20% between 2010 and 2030. 

 

But the occasion of the COVID pandemic has become a good excuse for policy analysts to propose their favorite solution to the crisis, indeed a “manufactured crisis” in the words of my friend and colleague David Berliner who introduced the term in 1995. Experts have offered no fewer than a dozen emergency measures needed for the nation’s recovery. Nine of these follow: 1. Smaller classes; 2. Tutoring; 3. Extending the school year; 4. Adding a fifth year of high school;  5. Focused funding on minority students; 6. Focusing on math, not reading; 7. Full-time summer school;  8. Increased teacher pay; 9. Focusing on students’ mental health. A fifth year of high school?! Seriously? Now I’m in favor of most of these things, except focusing on math. How about Civics? 

 

Some seem to favor no solution at all. A leitmotif of the NYT article is that the federal government’s billions of dollars in pandemic emergency aid to schools were misspent. The trouble is, less will be heard of any of these “solutioins” after NAEP scores wiggle up a couple of points in the next testing. 

   

The ravages of the pandemic were caused by a White House of Dunces. Incompetence cost the nation 1.1 million lives, two or three times more than the irreducible number. It will never happen again — at least not with COVID. The stewardship of all children’s education is a responsibility of every citizen, regardless of the next year’s tweak in NAEP scores. Put that on NAEP’s next Civics test.

 

Gene V Glass

Emeritus Regents’ Professor

Arizona State University

Categories Disruption, Health, NAEP, Standardized Testing
« Older Entries