Archives for category: Standardized Testing
« Older Entries

John Thompson: The Cruelty of Doubling Down on the Big Standardized Test

By dianeravitch
May 7, 2024 //
12

John Thompson, historian and retired teacher, reports on the cruelty of Oklahoma’s new crackdown on test scores.

He writes:

I wonder how most teachers responded to Nuria Martinez-Keel’s Tulsa Public Schools Ups the Intensity to Prepare for High Stakes Testing. I’m confident that few educators would be surprised by the language used by those who are implementing Oklahoma State Superintendent Ryan Walters’ teach-to-the-test program.  But, how many would believe that fellow educators really believe it will work? 

Martinez-Keel reports that “Angie Teas, a Tulsa Public Schools (TPS) instructional leadership director spoke positively about “a renewed focus on both academic standards and preparing students to take the standardized exams.” I was struck, however, by Teas’ words, “state tests are ‘part of our lives’ every year in public schools, but this testing season is ‘important for its own outside reasons.’” 

It seems likely that the “outside reasons” she cites are Superintendent Walters’ threats to takeover the TPS, as well as his order to immediately “elevate 12 of its schools off of the ‘F’ list.” In response to this seemingly impossible target, “the district provided high-dosage tutoring to 470 fourth and fifth graders, launched a campaign to combat chronic absenteeism and focused on credit-deficient seniors at struggling high schools to boost graduation rates, among other initiatives district leaders highlighted.” (A total of 1,125 elementary students will be served in a district with more than 33,000 enrollees.) 

I suspect that most educators would be supportive of efforts to assist small numbers of students if those policies were disconnected from test results, and they did not have the potential to undermine meaningful teaching and learning in the district as a whole. I find it hard to believe that teachers  who saw the harm inflicted by No Child Left Behind (NCLB) and Race to the Top (RttT) on students would not recognize why this new, stress-driven approach is also doomed to fail.

The narrative of educators in the story about upping the intensity of high-stakes testing is interesting, and it reminds me of the scripts my administrators would use as my colleagues repeatedly shouted back when ordered to do what virtually everyone saw as educational malpractice.  As Martinez-Keel reported:

Burroughs Elementary is one of the schools identified for improvement. Principal Dee Tisdale said the school has added academic rigor, focused on testing data and added extra resources, and it “all ties back” to individualized, small-group instruction between students and their teacher.

With state testing only days away, the mentality at Burroughs is “now it’s showtime.”

“I think in terms of the big championship game,” Tisdale said. “We’re just preparing, and we’re hoping that all of our practices will give us the trophy in June or July when the results come back.”

And as the TPS instructional leadership director Angie Teas said:

It’s nerve-wracking to feel the pressure of, ‘Oh my God, it feels like the world is watching,’ yeah,” … “But it’s also exciting to recognize that we’ve had an opportunity, like with OTEP (Oklahoma Teacher Empowerment Program) to be more all-parts-equal to the entire whole. We all see our part in the district that in a way I don’t think we have in a really long time.”

Martinez-Keel also cited an elementary teacher:

She felt ‘a little bit of pressure’ to make academic gains in only six small-group sessions. [Charity] Hargrave teaches fourth grade at Skelly but, through the OTEP program, was assigned 27 fifth graders at her school for extra instruction.

She said she had a ‘very short period of time’ to review benchmark test scores for each student, group them based on their performance level and plan lessons for each session.

But the experience has been positive, she said, and she hopes it will continue in the future.

Similarly, Asriel Teegarden said “Sometimes, there’s a little bit of fear about the unknown.” After a “30-minute lesson – featuring a space-themed reading comprehension exercise,” she asks her students “how did they feel about the ‘big test’ next week?” Then, “Teegarden said there’s a ‘different intensity level’ ahead of the most important testing period of her career.” But then she concluded: 

Usually, I would be nervous for these children, but I’ve gone about it like, ‘I’m excited you’re going to take this because you’re going to all do great,’… ‘Everything has got to be positive, giving them a lot of positive feedback. I think they’re going to do excellent.’

Of course, that sounds like wishful thinking to me. But I used to engage in a forced optimism in order to remain in the classroom and serve my students. Being an award-winning, veteran teacher who was successfully engaged in meaningful, challenging instruction, I was able to negotiate compromises with my principals who knew I would not participate in teach-to-the-test. I even agreed to an experiment in teaching a class in a way that would raise test scores without undermining my students’ meaningful learning. We produced the school’s highest History test results, but despite our best efforts, the stress overtook both my students and me. I swore to never again let testing influence our lessons.

Holistic instruction became more difficult over the years when my students from the poorest elementary schools volunteered that they had been “completely robbed of an education” by test prep.

Then, I came back from retirement to teach at an alternative charter school that was like my first school which served students with felony convictions. But then the top administrator ordered the principal to order me to focus solely on the few who had a chance to pass the Common Core high stakes tests. My principal asked me to briefly comply while she tried to persuade the district administration to cancel that plan. She said that they didn’t understand that this year’s end-of-instruction-exam was just a pilot, and thus wasn’t a graduation requirement. She was confident she could persuade them to withdraw the order to just assign most students worksheets, and focus solely on 3, 4, or 5 students per class.

Of course, my students were horrified by the new system, but they trusted me to not follow those rules except for a brief time.  However, when my principal apologized and said that the new system had to become permanent, she knew I’d resign, and I did.

So, I won’t criticize the 45 teachers who are each trying to help 25 or so students in the hope that they will benefit. I also appreciate journalists who are reporting on the stress being inflicted on teachers, and whether the data-driven, rushed interventions will somehow produce more good than harm. But I hope students will be their new priority as they review the research and the history of the failure of these sorts of mandates. And above all, I hope we will listen to our kids as the stress of test-prep is added to the stress of poverty, and attending high-poverty schools that are under attack by Ryan Walters.   

Categories Cruelty, Extremism, Oklahoma, Standardized Testing

Tom Ultican: One Cheer for NCLB…But Not From Me!

By dianeravitch
May 4, 2024 //
39

Tom Ultican has noticed a strange phenomenon on billionaire-funded websites, particularly The 74: Praise for the justly-reviled No Child Left Behind.

Teachers hated it because of its warped emphasis on standardized test scores. Students hated it because they were cheated of a real education, they lost civics, the arts, and recess, and the tests assumed more importance than they deserved.

But Ultican writes, Chad Aldeman of The 74 is nostalgic for the good old days of NCLB.

Neoliberals joined with libertarians to “reform”public education. Their tools were big money and propaganda distributed by media outlets like The 74, support by The Walton family (EIN 13-3441466) and Bill Gates (EIN 56-2618866). This year, regular columnist for The 74, Chad Aldeman, is trying to claim that lifting No Child Left Behind (NCLB) school accountability sanctions is responsible for the public school testing “data decline”.

In a recent article in The 74, Aldeman complained of widening achievement gaps in Indiana, but Ultican can’t find the source of Aldeman’s data.

Ultican notes that NCLB interrupted a long period of academic improvement.

From 1970 to 1992, America’s schools showed slow but steady improvement in education-testing outcomes but since the era of standards, testing and accountability, improvement basically stopped. Education, run by billionaires and politicians instead of educators, failed to improve testing outcomes.

Alderman stated in his latest article that it is not just an Indiana problem but that “49 of 50 states, the District of Columbia and 17 out of 20 of the large cities that participated in NAEP … saw a widening of their achievement gap over the last decade.” He did not share which tests showed widened achievement gaps nor which cohorts were compared. NAEP reports on reading scores for 4th and 8th grade do not show a significant change in scoring gaps between Black and White students and comparisons in other ethnic groups also were steady.

After asking what has caused this (non-existent) achievement gap increase, Alderman posited several possible reasons: Common Core state standards (CCSS), per-pupil spending, technology and social media. He said the timing for CCSS fit but did not explain why states where CCSS was never adopted had the same problem. For per-pupil spending, he claimed that more money was getting to classrooms, which defies education-spending reports, making his claim a little shady. For technology and social media, he said other countries with similar problems, did not see testing declines … a declaration made with no evidence cited.

If this decline were real, wouldn’t the privatization of public education be the most likely culprit? Charter schools came first followed by vouchers and more charter schools. Data clearly shows that vouchers harm student-testing performance. Furthermore both charter schools and voucher schools leech money from public education budgets.

He finally made his real point, “I argue that the weakening of school accountability pressures after the No Child Left Behind Act was passed is responsible for a large portion of the drop.” Those of us, who were in classrooms and witnessed the test-and-punish philosophy damage to public education, disagree. How many great public schools were labeled “failures and closed” because they existed in low income zip codes?…

Ultican concludes:

The 74 was founded in 2015 by former CNN news anchor, Campbell Brown, along with Michael Bloomberg’s education advisor, Romy Drucker. Its original funding came from the Dick and Betsy DeVos Family Foundation,  Walton Family Foundation, Doris and Donald Fisher Fund and Bloomberg Philanthropies. Since then, it has been the vehicle for spreading the billionaire message of privatization and undermining public schools.

Some billionaires see the non-sectarian nature of public education as a threat to their dreams of a Christian theocracy. Others are libertarians that oppose free universal public education, believing everyone should pay one’s own way and not steal people’s private properties using taxation. The Neoliberals are convinced that education should be run like a business and react to market forces.

Responding to the mission of The 74, Chad Aldeman’s series of articles, like those of many of his colleagues, are pure propaganda, shaping data to support his neoliberal ideology instead of honestly reporting facts. Unfortunately this kind of fake “journalism” is flooding email boxes and web pages throughout America every day.

Categories Accountability, Education Industry, Failure, Indiana, NCLB (No Child Left Behind), Standardized Testing

David C. Berliner and Gene V. Glass Review “Slaying Goliath”

By dianeravitch
April 9, 2024 //
15

I am almost four years late in discovering this review by two scholars for whom I have the greatest respect: David C. Berliner and Gene V. Glass.

I was happy to read this review because Slaying Goliath had a checkered fate. It was published in mid-January 2020. I went on a book tour, starting in Seattle. By mid-February, I made my last stop in West Virginia, where I met with teachers and celebrated the two-year anniversary of their strike, which shut down every school in the state.

As I traveled, news emerged of a dangerous “flu” that was rapidly spreading. It was COVID; by mid-March, the country was shutting down. No one wanted to read about the fight to save public schools or about its heroes. The news shifted, as it should have, to the panicked response to COVID, to the deaths of good people, to the overwhelmed hospitals and their overworked staff.

To make matters worse, the New York Times Book Review published a very negative review by someone who admired the “education reform” movement that I criticized. I thought of writing a letter to the editor but quickly dropped the idea. I wrote and rewrote my response to the review in my head, but not on paper.

Then, again by happenstance, I discovered that Bob Shepherd had reviewed the review of my book in The New York Times. He said everything that I wish I could have said but didn’t. His review was balm for my soul. Shepherd lacerated the tone and substance of the review, calling it an “uniformed, vituperative, shallow, amateurish ‘review.’” Which it was. His review of the review was so powerful that I will post it next.

Then, a few weeks ago, I found this review by Berliner and Glass.

The review begins:

Reviewed by Gene V Glass and David C. Berliner Arizona State University, United States

They wrote:

In a Post-Truth era, one must consider the source. 

In this case, the source is Diane Rose Silvers, the third of eight children of Walter Silverstein, a high school drop-out, and Ann Katz, a high school graduate. The Silvers were a middle-class Houston family, proprietors of a liquor store, and loyal supporters of FDR.

After graduation from San Jacinto High School, she enrolled in Wellesley College in September, 1956. Working as a “copy boy”for the Washington Post, Diane met Richard Ravitch, a lawyer working in the federal government and son of a prominent New York City family. They married on June 26,1960, in Houston, two weeks after Diane’s graduation from Wellesley. The couple settled in New York City, where Richard took employment in the family construction business. He eventually served as head of the Metropolitan Transit Authority and Lieutenant Governor in the 2000s, having been appointed by Democratic Governor David Paterson.

 Diane bore three sons, two of whom survived to adulthood. Diane and Richard ended their 26-year marriage in 1986. She had not been idle. For a period starting in 1961, Diane was employed by The New Leader, a liberal, anti-communist journal. She later earned a PhD in history of education from Columbia in 1975 under the mentorship of Lawrence Cremin.

Diane was appointed to the office of Assistant Secretary of Education, in the Department of Education by George H. W. Bush and later by Bill Clinton. In 1997, Clinton appointed her to the National Assessment Governing Board (NAGB), on which she served until 2004. 

Ravitch worked “… for many years in some of the nation’s leading conservative think tanks.

Read the full pdf here.

ravitch-reviewDownload
Categories Accountability, Betsy DeVos, Billionaires, Broad Foundation, Charter Schools, Common Core, Corporate Reform, Curriculum, Democracy, Disruption, Education Industry, Ethics, Failure, Funding, Gates Foundation, Bill Gates, History, NCLB (No Child Left Behind), Parents, Privatization, Ravitch, Diane, Resistance, School Choice, Standardized Testing, Students, Teacher Evaluations, Teachers, Walton Foundation

New York Will Move Away from Teacher Evaluation Tied to Test Scores

By dianeravitch
April 8, 2024 //
48

Back in 2010, Secretary of Education Arne Duncan rolled out his Race to the Top program to reform American education. The U.S. Department of Education offered a total of $5 billion to states. To be eligible to compete for a part of the huge prize money, states had to agree to authorize charter schools, to adopt the Common Core (not yet finished), and to evaluate teachers based on the test scores of their students.

The requirement to change teacher evaluation was heated. Duncan scoffed at critics, saying they were trying to protect bad teachers and didn’t want to know the truth.

Debate over this methodology was heated.

I was part of a group of education scholars who denounced this method of evaluating teachers in 2010.

In 2012, three noted scholars claimed that teachers who raised test scores raised students’ lifetime incomes; President Obama cited this study, led by Harvard economist Raj Chetty, in his State of the Union address. It seemed to be settled wisdom that teachers who raised test scores were great, and teachers who did not should be ousted.

In 2014, the American Statistical Association warned about the danger of evaluating teachers by the test scores of their students. The ASA statement said that most studies of this method find that teachers account for 1-14% of the variation in test scores. The greatest opportunity for improvement, they said, was to be found in system-level changes.

The Gates Foundation poured hundreds of millions of dollars into districts willing to test value-added methodology, and eventually gave up. Teachers were demoralized, teachers avoided teaching in low-income districts. Overall improvements were hard to find.

Arne Duncan was a true believer, as was his successor, John King, and they never were willing to admit failure.

Teachers never liked VAM. They knew that it encouraged teaching to the test. They knew that teachers in affluent districts would get higher scores than those in less fortunate districts. Sometimes they sued and won. But in most states, teachers continued to be evaluated in part by their students’ scores.

But in New York state, the era of VAM is finished. Dr. Betty Rosa, the chancellor of the New York State Board of Regents, reached an agreement with Melinda Person, president of New York State United Teachers, to draft a new way of evaluating teachers that moves away from students’ standardized test scores.

New York state education leaders and the teachers’ union have announced an agreement to change how New York school teachers and principals are evaluated, and move away from the mandated reliance on standardized test scores.

State Education Department Commissioner Betty Rosa and New York State United Teachers President Melinda Person hand-delivered their drafted legislation Wednesday to lawmakers to create a new system that doesn’t use students’ test performance to penalize educators. The state teacher evaluation system, known as the Annual Professional Performance Review, or APPR, was modified in the 2015 budget to place a greater importance on scores.

“It’s connecting research to practice and developing strategies to ensure that teachers have the best tools and principals to make sure our young people are getting the best quality education,” Rosa told reporters Wednesday in the Legislative Office Building.

When NYSUT elected president Person last year, she said her first task was to change the teacher evaluation system, and state lawmakers said with confidence Wednesday it will happen this session.

The proposed law, which has not officially been introduced in the Legislature, would remove the requirement to base evaluations on high-stakes tests. School districts would have eight years to transition, but could make the changes faster than the required deadline.

Person argued it will support new teachers who are often burdened by the required paperwork under the current model.

“This would be a fair and a just system that would support them in becoming better educators, which is ultimately what they want to do anyway,” Person said.

The proposal was negotiated in agreement with state superintendents, principals, school boards, the PTA, Conference of Big 5 School Districts and other stakeholders. The issue has been contentious for union and education leaders for years, and both state Education Committee chairs in the Legislature said they’re thrilled with the agreement. 

“That’s such a nice thing in Albany,” said Senate Education chair Shelley Mayer, a Democrat from Yonkers. “Who can do that? Who gets agreement? It’s very hard around here.

“It takes a woman to do it,” Assembly Education chair Michael Benedetto replied with a smile.

Benedetto, a Bronx Democrat, was a classroom teacher for decades and recalled how feedback helps educators develop when done in the proper way.

“It’s like anything else — we want stability in our lives, we want to know where we’re going, how we’re going to be rated and what we’re going to be rated on, as a teacher, as a professional,” the assemblyman said.

Lawmakers will review the proposal and draft legislation in the coming weeks.

Remembering how strident were the supporters of VAM, it’s kind of wonderful to hear the collective sigh of relief in Albany as it fades away.

Categories Accountability, Charter Schools, Corporate Reform, Disruption, Duncan, Arne, Education Industry, Education Reform, Failure, Gates Foundation, Bill Gates, Standardized Testing, Teacher Evaluations, VAM (value-added modeling)

Paul Thomas: Big Lies of Education: Confusing NAEP Proficient with “Grade Level”

By dianeravitch
February 28, 2024 //
25

I served on the National Assessment Governing Board from 1998-2004. NAGB is the governing agency for NAEP, the federal test. I was appointed by President Bill Clinton. I learned about the inner workings of standardized testing, much of which made me skeptical of it.

I have often observed that critics of public schools assume that NAEP Proficient is the same as “grade level,” when in fact NAEP warns readers explicitly in every score report that NAEP Proficient is NOT “grade level.” In fact, NAEP Proficient represents mastery of what was tested, which I would characterize as an A or A-.

In 2010, when the anti-public school documentary “Waiting for ‘Superman’” was released, I reviewed it for The New York Review of Books and criticized it for confusing NAEP Proficiency with grade level, then claiming that most American kids can’t read, all because of their terrible public schools, their terrible teachers and those awful unions. The way to a better future, the documentary claimed, was charter schools. Not true. Even Secretary of Education Miguel Cardona has repeated this erroneous claim. Apparently neither he nor his speech writers reads NAEP reports with care and no one has briefed them.

I have explained this confusion on several occasions on the blog. I even called the Commissioner of the National Center on Education Statistics and proposed that NAEP Proficient be renamed “NAEP Mastery,” to clarify its meaning. She sounded enthusiastic about the idea (which came from a reader of this blog) but nothing changed.

I am very happy to see that Professor Paul Thomas at Furman University in South Carolina has launched a series called “Big lies in Education,” and this claim is one of the Big Lies. It is a lie because the fact that NAEP Proficient is not grade level is stated plainly in every release of NAEP scores.

Thomas begins:

One of the most bearish statistics for the future of the United States is this: Two-thirds of fourth graders in the United States are not proficient in reading,” wrote Nicholas Kristof in The New York Times.

Kristof’s piece in 2023 can be traced back to a similar claim by Emily Hanford in 2018: “More than 60 percent of American fourth-graders are not proficient readers, according to the National Assessment of Educational Progress, and it’s been that way since testing began in the 1990s,” including a surprisingly ineffective graphic.

Open the link to see this and other graphics.

The student reading proficiency Big Lie grounded in misrepresenting or misunderstanding NAEP is likely one of the most complicated Big Lies of Education.

In media and political rhetoric, first, the terms “reading proficiency” and “grade level reading” are commonly jumbled and used inappropriately as synonyms.

Achievement levels such as “basic” and “proficient,” such as used in NAEP for reading, are misleading and complicated for most people not familiar with technical terminology.

NAEP “basic” is approximately grade level (although even that claim is problematic since no standard exists in the US for “proficient” or “grade level”), and “proficient” on NAEP is high: 

Another important graph. Open the link.

Hanford’s and Kristof’s Big Lie, then, is a combination of blurring NAEP achievement levels with grade level reading achievement and manufacturing a reading crisis with that misinformation.

Ironically, NAEP grade 4 reading scores for a decade show that 2/3 of students are reading at or above grade level, the inverse of the false crisis claims of the media:

Open the link for the graph.

This is an excellent expose, which everyone should read. The claim that most kids read below grade level is foundational to the claim that public schools are in crisis. Its a Big Lie.

Categories Accountability, Corporate Reform, Fake, Lies, Literacy, NAEP, Standardized Testing

What Do Test Scores Measure? How I Lost My Faith in Their Value

By dianeravitch
February 14, 2024 //
58

For many years, I was a staunch advocate of standardized testing. But I lost my enthusiasm for standardized testing after spending seven years on the governing board of NAEP (the National Assessment of Educational Progress). NAEP is the federal test administered every two years to measure academic progress in reading and math, as well as testing other subjects. The test takers are randomly selected; not every student answers the questions on any test. There are no stakes attached to NAEP scores for any student, teacher, or school. The scores are reported nationally and by state and for nearly two dozen urban districts. NAEP is useful for gauging trends.

Why did I lose faith in the value of standardized testing?

First, over the course of my term, I saw questions that had more than one right answer. A thoughtful student might easily select the “wrong” answer. I also saw questions where the “right” answer was wrong.

Second, it troubled me that test scores were so highly correlated with socioeconomic status. Invariably, the students from families with the highest income had the highest scores. Those from the poorest families had the lowest scores.

Third, the latter observation spurred me to look at this correlation between family wealth and test scores. I saw it on the results of every standardized test, be it the SAT, the ACT, or international tests. I wondered why we were spending so much money to tell us what we already knew: rich kids have better medical care, fewer absences, better nutrition, more secure and stable housing, and are less likely to be exposed to vermin, violence, and other health hazards.

Fourth, when I read books like Daniel Koretz’s “Measuring Up” and “The Testing Charade” and Todd Farley’s “Making the Grades: My Misadventures in the Standardized Testing Industry,” my faith in the tests dissipated to the vanishing point.

Fifth, when I realized that the results of the tests are not available until the late summer or fall when the student has a new teacher, and that the tests offer no diagnostic information because the questions and answers are top-secret, I concluded that the tests had no value. They were akin to a medical test whose result is available four months after you see the doctor, and whose result is a rating comparing you to others but utterly lacking in diagnostic information about what needs medication.

So, all of this is background to presenting a recent study that you might find useful in assessing the value of standardized tests:

Jamil Maroun and Christopher Tienken have written a paper that will help you understand why standardized tested is fatally flawed. The paper is on the web and its title is:

The Pernicious Predictability of State-Mandated Tests of Academic Achievement in the United States

Here is the abstract:

The purpose of this study was to determine the predictiveness of community and family demographic variables related to the development of student academic background knowledge on the percentage of students who pass a state-mandated, commercially prepared, standardized Algebra 1 test in the state of New Jersey, USA. This explanatory, cross-sectional study utilized quantitative methods through hierarchical regression analysis. The results suggest that family demographic variables found in the United States Census data related to the development of student academic background knowledge predicted 75 percent of schools in which students achieved a passing score on a state standardized high school assessment of Algebra 1. We can conclude that construct-irrelevant variance, influenced in part by student background knowledge, can be used to predict standardized test results. The results call into question the use of standardized tests as tools for policy makers and educational leaders to accurately judge student learning or school quality.

The paper was peer-reviewed. It was published last week.

Categories Accountability, Education Industry, Education Reform, NAEP, Standardized Testing

Yong Zhao: Why Doesn’t the U.S. Scrap PISA?

By dianeravitch
February 12, 2024 //
49

When I learned that the latest PISA (Program on International Student Assessment had been released, I attended a webinar, where I learned once again that the scores of U.S. 15-year-old students were somewhat below the international average. The PISA tests in math, reading, and science have been offered since 2000, sponsored by the Organization for Economic Co-operation and Development.

My takeaway from the webinar was that we should try to be more like Singapore and Macau.

I have studied the results of international assessments such as PISA and TIMSS for years. Eventually, I began to wonder what the connection was—if any—between the test scores of 15-year-old students and the economic productivity of their nation 10, 15, 20 years later. We’ve been bemoaning our scores since the first international tests were given in the 1960s, even as our economy soars way beyond the nations with higher scores on the tests.

I invited Yong Zhao to share his reaction to the latest PISA scores. His response was as brilliant as I anticipated.

Yong Zhao is one our most accomplished scholars of education. Born in China to an impoverished family, he pursued his dreams, migrated to the United States, and has made his mark as a creative and innovative thinker. He is currently a Foundation Distinguished Professor of Education at the University of Kansas and holds an appointment as Professor of Educational Leadership at the University of Melbourne. His list of honors and publications is too long for me to recite here. But you can find it online.

Yong Zhao wrote:

It doesn’t make sense: Why Is the US Still Taking the PISA?

I have always wondered what America has got from participating the PISA every three years. Since 2000, the U.S. has been taking part in this nonsensical global academic horse race. Every time it took the test, American students stood at about the middle of the global league table. Every time the results were released, American media would point out how American students are not the best, but East Asian education systems such as China, Hong Kong, Chinese Taipei, South Korea, Japan, and Singapore are the best. And then U.S. authorities would invite PISA and other pundits to tell us how to improve American education.

The same story has been going on for more than two decades, but American education has not improved, at least according to the PISA scores. According to the most recent results (NCES, 2023), American students did much worse in math in 2022 than in 2003, with an 18-point decline from 483 to 465. Their reading and science scores, however, remained about the same without significant change over the past two decades. Although PISA experts largely blame the COVID pandemic as the reason for the decline in math, it does not make much sense because there is no decline in reading and science. Did COVID-19 only affect math, not science and reading? Of course, one can try to argue that reading and science are much less sensitive to COVID, but why? 

Basically, the international standing of the US and the test scores of its students have not changed much. Whatever the PISA data revealed and/or the lessons from other countries such as China, Japan, Singapore, or Finland have not helped improve America’s PISA scores. By the way, Finland, the country Americans view with the best education system because of its early stunning PISA performance, has seen a much more dramatic decline in its PISA scores: from 544 to 484—a 60-point decline in math, from 546 to 490—a 56-point decline in reading, and from 563 to 511—a 52-point decline over the past two decades. Not sure if America still views Finland as the best education country, but its scores have dropped to almost the same point as American students. 

In fact, other than Finland, the PISA league tables have not changed much either. East Asian education systems have consistently remained the top performers and the OECD countries’ average scores have been dropping. If PISA had any impact on the world’s education quality and equity, education should not be the same as 20 years ago.

PISA does not really have much to offer to anyone, except those who benefit from the test itself—the consultants, the test makers, the data processors, and possibly some education politicians.

In a review article (Zhao, 2020), I summarized the research about  PISA and found: 1) PISA markets itself as an assessment of abilities needed in the 21st Century, but it is the same as other international tests such as TIMSS, 2) PISA ignores the overall educational purposes of different countries by primarily assessing math, reading, and science, 3) PISA’s tests are not of high quality with numerous theoretical and technical problems, and 4) PISA’s sampling has been manipulated in different countries. My conclusion is that instead of bringing positive changes to the world, PISA wreaked havoc.

America has never excelled in international tests since the beginning of such assessment in the 1960s, but the low scores have not seemed to affect it much. In fact, a correlational analysis done in 2007 showed a negative correlation between international test scores and economic development (Baker, 2007). That is, countries with higher scores in the first international study did worse than countries with lower scores. If PISA or any other international tests truly measure what matters in education, America should no longer be a developed country. On the contrary, East Asian countries have always scored well in international assessments, but their economic development has been more related to economic, political, and international orders than their test scores.

What matters to economic development and prosperity is perhaps the non-cognitive factors that PISA does not typically emphasize. For example, in an analysis, I found that PISA scores are negatively correlated with entrepreneurship confidence across countries (Zhao, 2012b). American students, despite their lower scores, have always had more confidence than their peers in other countries. In fact, confidence has been found to have negative correlations with test scores (Zhao, 2012b, 2014, 2018b). High score education systems, except Finland, have always had a negative impact on students’ social and emotional wellbeing (Zhao, 2012a). Even PISA’s own data show that PISA scores are negatively correlated with life satisfaction of students (OECD, 2019).

Many education systems participate in PISA because they are fooled by its claim to measure global competitiveness. Somehow these educational systems are convinced that their PISA scores and rankings mean how competitive they are globally. But this is not true and cannot be true. In 2022, over 80 education systems took part in the PISA but these systems are hugely different. For example, the U.S. has three hundred million people and does not really have an education system (it has over 50 education systems based on the number of states and over 12,000 systems if we treat each school district as a system). How can it be compared with Macao, China, a tiny place with about 688,000 people and one education system? Likewise, how can the U.S., with a per capita GDP of over $70,000 be compared with Albania, whose per capita GDP is about $6,000.

Moreover, PISA has been operational for over 20 years. The first cohort of 15-year-old students took the test in 2000. If PISA truly has predictive power, it should have produced a longitudinal study to show how these students do in society. They are about 39 years old today. But we haven’t seen any such report except the wild guesses made by some scholars (Hanushek & Woessmann, 2010).

If PISA offers nothing, why does the U.S. spend the money and effort to join the game? For monitoring of basic education conditions, it already has the National Assessment of Education Progress (NAEP) or the national report card, which has been in existence since 1969. Why continue to participate in PISA?

Frankly, it’s inexplicable, for there is truly no reason the U.S. should continue to participate in PISA, let alone to pretend to learn from high performing countries. The lessons PISA offered have not been productive. For example, the lesson that high performing systems (e.g., Singapore, South Korea, and Finland) recruit high performing high school graduates to be teachers (Barber & Mourshed, 2007) is not based on real evidence and does not really produce better education outcomes (Gronqvist & Vlachos, 2008). The lesson that high performing systems have clear definitions of learning expectations, a good structure of different stages, and tough measures to ensure that students have met the expectations (Tucker, 2011) is intended largely to copy East Asian education systems; but, ironically, the East Asian countries have been working very hard to change these practices (Zhao, 2014). International learning may make sense sometimes, but there are great limitations (Zhao, 2018a). American education should focus on developing its own way to improve education instead of trying to catch up with others (Zhao, 2009). 

This is not to say that American education is perfect. Rather, it is to say the way forward is not to look at what others have been doing. The U.S. needs to solve its own problems and work on creating a better future. With the emergence of ChatGPT and other generative AI tools, the world has changed again. If ChatGPT had taken the 2022 PISA, it is highly likely that it would outscore all the students in the world. It would be the best education system accordingly. Today, many students use AI tools to do their schoolwork, and teachers use AI in their teaching. PISA has become even more irrelevant.

Since 2000, our scores on PISA have barely changed. While there’s much chatter about learning from other systems, it has not happened. There is no reason that the U.S. should continue its participation in PISA.

References:

Baker, K. (2007). Are International Tests Worth Anything? Phi Delta Kappan, 89(2), 101-104. 

Barber, M., & Mourshed, M. (2007). How the World’s Best-Performing School Systems Come out on Top. Retrieved from New York: https://www.mckinsey.com/industries/social-sector/our-insights/how-the-worlds-best-performing-school-systems-come-out-on-top

Gronqvist, E., & Vlachos, J. (2008). One size fits all? The effects of teacher cognitive and non-cognitive abilities on student achievement. Retrieved from Stockholm, Sweden: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=1311222

Hanushek, E. A., & Woessmann, L. (2010). The High Cost of Low Educational Performance: The Long-run Economic Impact of Improving PISA Outcomes. Retrieved from Paris: http://books.google.com/books?id=k7AGPo0NvfYC&pg=PA33&lpg=PA33&dq=hanushek+pisa+gdp&source=bl&ots=2gCfzF-f1_&sig=wwe0XLL5EblVWK9e7RJfb5MyhIU&hl=en&sa=X&ei=MLPCUqaOD8-JogS6v4C4Bw&ved=0CGcQ6AEwBjgK#v=onepage&q=hanushek%20pisa%20gdp&f=false

NCES. (2023). Program for International Student Assessment (PISA). Retrieved from https://nces.ed.gov/surveys/pisa/index.asp

OECD. (2019). PISA 2018 Results (Volume III): What School Life Means for Students’ Lives. Retrieved from https://doi.org/10.1787/acd78851-en.

Tucker, M. (Ed.) (2011). Surpassing Shanghai: An Agenda for American Education Built on the World’s Leading Systems. Boston: Harvard Education Press.

Zhao, Y. (2009). Catching Up or Leading the Way: American Education in the Age of Globalization. Alexandria, VA: ASCD.

Zhao, Y. (2012a, December 11). Numbers Can Lie: What TIMSS and PISA Truly Tell Us, if Anything?  Retrieved from http://zhaolearning.com/2012/12/11/numbers-can-lie-what-timss-and-pisa-truly-tell-us-if-anything/

Zhao, Y. (2012b). World Class Learners: Educating Creative and Entrepreneurial Students. Thousand Oaks, CA: Corwin.

Zhao, Y. (2014). Who’s Afraid of the Big Bad Dragon: Why China has the Best (and Worst) Education System in the World. San Francisco: Jossey-Bass.

Zhao, Y. (2018a). Shifting the Education Paradigm: Why International Borrowing Is No Longer Sufficient for Improving Education in China. ECNU Review of Education, 1(1), 76-106. 

Zhao, Y. (2018b). What Works May Hurt: Side Effects in Education. New York: Teachers College Press.

Zhao, Y. (2020). Two decades of havoc: A synthesis of criticism against PISA. Journal of Educational Change, 1-22. doi:10.1007/s10833-019-09367-x

Categories Accountability, Education Industry, International, Standardized Testing

North Carolina: More $$$ for Voucher Schools, More Tests for Public Schools

By dianeravitch
February 6, 2024 //
3

It is a strange irony that Red State legislators believe passionately in testing and accountability for public schools, yet excuse voucher schools from those same measures. If public school students must be tested, why not students who receive vouchers to attend private and religious schools on the taxpayers’ dime? Why not use the same measuring stick for all students so the voucher schools can be held accountable?

Here is a report from Public Schools First NC:

North Carolina’s voucher program has been widely criticized for its lack of accountability. The Opportunity Scholarship and ESA+ programs come with little financial oversight, no curriculum or content standards requirements, no educational or credential requirements for teachers, and no publicly available student performance testing data.

Since 1992, NC students in grade 3-8 public schools take the North Carolina End-of-Grade (EOG) Tests designed to measure student performance on the goals, objectives, and grade-level competencies specified in the NC Standard Course of Study. Standards in NC are used at the state level to ensure all students will be taught the content deemed essential and necessary by the state to allow teachers and parents to assess student progress and readiness for the next grade level.

The 2023 Appropriations Act takes a small step toward addressing the gap between the abundant regulation and accountability measures in place for traditional public schools and the state’s laissez faire approach to private schools. After spending half a billion dollars since 2014 and adding hundreds of millions of dollars to the voucher program’s annual budget this year, legislators added a small testing provision that may enable taxpayers to get a glimpse into the academic performance of voucher recipients. Starting in 2024-25, public school students and voucher recipients in grades 3, 8, and 11 will be administered the same standardized test.

But instead of requiring the estimated 4,000 grade 3 and 8 voucher-receiving private school students to take NC’s EOG tests, the approximately 220,000 public school students in grades 3 and 8 will have to take a nationally standardized test that was not developed to measure NC Standard Course of Study in addition to taking the EOG.

The common test in 11th grade will be the ACT, which is already administered to public school students statewide. Currently, voucher-receiving private schools are required to take a test (selected by the school administrator) in 11th grade. Many private schools already administer an assessment of their choice, so there will be little change other than NOW the state is picking up the tab for the new tests.

The common test for grades 3 and 8 will be recommended by the Superintendent of Public Instruction. But the 2023 Appropriations Act (p. 193) specifies that it must be a nationally standardized test, which disqualifies the NC End-of-Grade (EOG) and End-of-Course (EOC) tests. Although they are standardized tests, the EOGs and EOCs are specifically designed to assess goals and objectives of the NC Standard Course of Study, not a broad swath of national standards. Because they are only administered to students in North Carolina, they aren’t “nationally” standardized.

The national tests, often called “off the shelf” tests, are designed to appeal to as many states and districts as possible, so they measure standards common across states rather than one specific state. As a result, they may not align well with the North Carolina Standard Course of Study.

The EOG and EOC test administrations are required by federal and state law, so there is no option to replace them with a test that doesn’t specifically measure the North Carolina Standard Course of Study. However, as a condition of receiving voucher funds, the General Assembly could require the 4,000 or so voucher-receiving students in grades 3 and 8 to take the EOG tests instead of adding another test for the 220,000 public school students in grades 3 and 8.

In addition to affecting the testing burden of far fewer students, administering the EOG to private school students would be much cheaper than purchasing an “off the shelf” national test for hundreds of thousands of public school students.

Should public school students have to take another standardized test to assure lawmakers that private school students are learning? Should taxpayers have to foot the bill for hundreds of thousands of new tests instead of paying next to nothing for a few thousand EOG tests that are already developed and administered in North Carolina?

The legislative short session starts in April. This testing provision may be changed if enough people encourage their legislators to address it. Contact your legislators.

Read our fact sheet for more information on student testing in North Carolina.

Categories Accountability, Education Industry, North Carolina, Republicans, Standardized Testing, Vouchers

Gary Rubinstein: Why Don’t Success Academy High School Students Take Advanced Regents Exams?

By dianeravitch
February 6, 2024 //
12

Gary Rubinstein has been following the ups and downs of New York’s highest scoring charter school chain: Success Academy. Every year, the grades 3-8 test scores at the chain are through the roof. But Gary noticed that the high school students at Success Academy do not take Advsnced a regents exams as they do at the New York City’s highest performing high schools.

Gary examines this question:

Success Academy is a charter network with about 40 schools in the New York City area. They are known for their high standardized 3-8 test scores. Though it has been proved that their test scores are somewhat inflated by their practices of shedding their low performing students over the year and also by, at some schools, focusing exclusively on test prep in the months leading up to the tests, they still have these test scores to show their funders and the various charter school cheerleaders.

In June there was an article on the website of something called Albany Strategic Advisors, some kind of consulting firm about how well middle school students at Success Academy performed on four of the New York State Regents exams: Algebra I, Living Environment, Global History, and English. The last sentence of the second to last paragraph explains that these results are important because “Taking the exams in middle school allows students to take more advanced college preparatory courses in high school.”

These ‘more advanced college preparatory courses in high school’ include 10 other courses that have Regents exams including Geometry, Algebra II, Chemistry, Physics, US History, and Spanish. The minimum requirements for getting what is called ‘a Regents diploma’ in New York is one math Regents, one science Regents, one Social Studies Regents, and the English Regents. But to get an ‘Advanced Regents diploma’ you need all three maths and all three sciences and one foreign language Regents. Most competitive high school have their students take these other Regents which are known to be fairly straight forward tests with very generous curves.

About 8 years ago I noticed that there were no Regents scores for any of the other 10 exams in the Success Academy high school. Then 6 years ago I found that some of their students actually were taking some of the more difficult Regents but they were doing very poorly on them. And now, 6 years later, I checked up on them again to find that in the three Success Academy high schools which enroll a total of about 1,100 students from grades 9 to 12, they again do not have any scores for any of the Regents that are typically taken at competitive schools.

So why does this matter?

Well, Success Academy has spent eighteen years carefully cultivating their image. They want families to think that they have the highest expectations and that families should trust them to educate their children because those higher expectations will lead to those students learning the most. And we all know about their 3-8 state tests in Math and ELA. But it is pretty ‘odd’ that their students don’t take the more difficult Regents. The most likely reason for this is that Success Academy only wants information public that makes them look good and avoids any action that could reveal public data that reveals that they do not live up to their reputation. So I believe that they don’t allow their students to take the Regents because they believe that the scores on those Regents won’t be as impressive as their 3-8 state test scores compared to other schools. If I am right then this is an example of Success Academy choosing to preserve their inflated reputation over giving their students the opportunity to challenge themselves on these competitive exams.

Please open the link to finish the article. Nobody does this kind of close review better than Gary Rubinstein.

Categories Accountability, Charter Schools, Education Industry, Harlem Success Academy, New York, New York City, Standardized Testing

Nancy Bailey: What Secretary Cardona Should Do Now

By dianeravitch
January 30, 2024 //
26

Secretary of Education Miguel Cardona has been nearly invisible these past three years, other than lamenting test scores. Veteran educator Nancy Bailey has some suggestions about how he could help kids, schools, and teachers right now. This post was reposted by the Network for Public Education.

The secretary keeps talking about raising the bar. Nancy Bailey has some thoughts about some bars he could work on. Reposted with permission.

She writes:

Education Secretary Cardona focuses on reducing absenteeism, tutoring, and after-school programs. And he refers to raising the bar, which sounds like A Nation at Risk talk.

Yet there are so many K12 issues that Cardona and the Biden administration could address, lead, and support the states and local school districts.

Here are some educational issues Cardona should drive this new year. If you have more, please share.

1. The Arts

Poor public schools have gone without the arts for years. Cardona should push for qualified art and music teachers for every school.

The arts help students struggling with mental health difficulties, and jobs exist in the arts.

Every child in K12 should have access to a vital arts program.

2. Career Education

Career-technical education is essential, but companies pushing their agendas into high schools to get workers raises concerns.

Tax dollars should help students decide what careers they want, giving them the chance to experience high school, not creating schools for corporate preparation.

3. Class Size

Reducing class sizes would help students with disabilities in inclusion classes and is essential for student safety. Cardona must endorse lowering class sizes and showcase schools that do.

If schools can’t lower every class (classes like P.E. wouldn’t be necessary), give students access to at least one small class where they are known.

Lowering class sizes in K-3rd grade would also help children get a good start. See the STAR Study.

4. Corporal Punishment

In 2023, The Washington Post reported that 15 states still permitted corporal punishment in schools (St. George, 2023). Like Florida, which vaguely gives a nod to it. Teens who wear the wrong kind of prom dress or misunderstandings resulting in paddling are examples.

Cardona deserves praise for standing against corporal punishment last year in schools, claiming educators should favor evidence-based approaches and that there should be no spanking, hitting, or paddling.

5. Data

Cardona must study and draw attention to child privacy laws which are currently inadequate.

In 2018, the NEPC published Educating All Our Children: Your Kids, Their Data, No Privacy by Linda McSpadden McNeil.

She stated:

When children’s privacy is violated in ways that are overt, visible, and knowable, the violation is unquestioned. It is unacceptable. In most cases, it is illegal.

So why is it different when the violation is hidden, opaque, electronic, commercial, and complicated?

6. High-Stakes Standardized Testing

President Biden promised teachers and parents he’d end harmful high-stakes standardized tests. Instead, he pushed assessments even during the pandemic.

Can Cardona show the American people who want these tests reduced or eliminated what, if anything, the Biden administration will finally do to end high-stakes standardized tests?

7. Lead in School Pipes

The Biden-Harris team promised to repair the lead pipes in homes and schools. They’re to be commended for this. It would be nice, however, if Cardona presented a report.

No amount of lead is O.K. for developing children, and it can lead to learning disabilities.

8. Reading

Reading has become a volatile issue, and Cardona has been mostly silent. Many commercial programs with little independent study, but nonetheless called the Science of Reading, are being pushed into classrooms.

The subject of reading deserves a better forum than short, often hostile debates on X.

Cardona should call for a new National Reading Panel to study programs and address reading instruction. The panel should include teachers who teach reading since they were left off the last panel. This panel should consider the concerns of the last NRP member, the only educator on the panel, Joanne Yatvin (Yatvin, 2002). Parents should also be given a voice on this panel. A new NRP would allow for a better collection of the research, old and the latest findings, and a review of the work of the last NRP.

9. Recess

Every public school in the nation should give children several unstructured breaks throughout the school day, supervised, on safe, well-designed playgrounds.

Driving students to work nonstop with mindfulness training or a dozen other excuses to bypass recess should not be permitted.

10. School Buildings

Americans need to know the status of their public school buildings. How are the HVAC systems and air quality? How many school facilities are still falling apart? Are public school buildings safe?

Are new schools being built to support teachers?

11. School Choice

The Biden administration discussed regulating charter schools (Lieberman, 2022). But concerns about vouchers, educational savings account, nonprofit and for-profit charters, and religion in schools needs to be better addressed.

In addition, the Biden administration should describe what they mean by community schools (often called charter schools), partnerships, and social impact bonds and how these schools are still public.

12. School Libraries and Librarians

It’s an abomination to drill children to learn to read in poor schools, and then not provide them a school library with a qualified school librarian.

Closing school libraries has been a disaster in many school districts, see Philadelphia as an example.

13. School Safety

The gun lobby is unfortunately influential, so despite shootings in this country, don’t hold your breath for meaningful gun laws. It was thoughtful of Cardona to visit Parkland.

In the meantime, school administrators must devise creative ways to make schools safe and identify students in their schools who need mental health help. See class size above.

14. Social-Emotional Learning

Concerns about the data collected on students includes social-emotional learning. Teachers always want their students to be socially and emotionally healthy, but social-emotional learning seems more about collecting unnecessary sensitive data about students.

Who is using this information and why?

15. Special Education

Parents are in danger of losing special education services. The All Handicapped Children’s Act reauthorized to IDEA should have evolved into a more inclusive and better programs for students with exceptional needs, but instead it has been hijacked by those who don’t want to pay for it.

Cardona should look into special education and create a task force to study it and determine exactly how much special education funding schools receive and the kind of services students receive. He might start with Tammy Kolbe’s work and the National Education Policy Center report Funding Special Education: Charting a Path that Confronts Complexity and Crafts Coherence.

16. Teacher Preparation

School reform has changed teacher preparation dramatically. There needs to be more effort to oversee these mostly fast-track nonprofit or for-profit programs often connected to charter schools.

Cardona should step up here to promote fully university-prepared teachers, and he should work with university education programs to improve their coursework and degree offerings. The United States Department of Education might provide scholarships to attract young people who want to pursue a teaching career in university education programs, not unknown nonprofits or charter school preparation.

Fast-track groups like Teach for America are a concern because they turn those without real qualifications into the system with little understanding of child development or what’s needed to teach well, and Grow Your Own programs are ill-defined.

17. Technology

While technology is useful to learning, a recent Columbia University study indicates that children read better with paper print not online.

The Guardian cites MIT neuroscientist John Gabrieli, skeptical about the promises of big tech and its salesmen.

Gabrieli states:

I am impressed how educational technology has had no effect on scale, on reading outcomes, on reading difficulties, on equity issues.

Is Cardona behind teachers or for replacing them with technology? Actions matter.

18. Third-Grade Retention

Third-grade retention is unnecessary. No child should be made to feel like a failure. Children can still learn to read in third grade, can still grow and become great learners. Speaking out on this issue would help end it.

References

St. George, D. (2023, August 10). In over 15 states, schools can still paddle students as punishment. The Washington Post. Retrieved from https://www.washingtonpost.com/education/2023/08/10/school-paddling-corporal-punishment/.

Yatvin, J. (2002). Babes in the Woods: The Wanderings of the National Reading Panel. Phi Delta Kappan, 83(5), 364–369. https://doi.org/10.1177/003172170208300509

Lieberman, M (2022). Lawmakers, Education Secretary Clash Over Charter School Rules. Education Week, https://www.edweek.org/policy-politics/lawmakers-education-secretary-clash-over-charter-school-rules/2022/04

Share this:

FacebookTwitterEmailShare

Readers wishing to comment on the content are encouraged to do so via the link to the original post.

Find the original post here:View original post

Categories Art, Arts Education, Biden, Cardona, Miguel, Class size, Corporate Reform, Creativity, Data, Education Reform, Funding, Literacy, Privacy and Privacy Rights, Privatization, School Choice, Standardized Testing, Teachers and Teaching
« Older Entries