Archives for category: NAEP
« Older Entries
Newer Entries »

The Absurdity of Expecting All Students to Reach NAEP Proficiency

By dianeravitch
March 28, 2018 //
21

 

James Harvey here explores “the problem with proficiency.”

Common Core tests arbitrarily decided that the NAEP proficiency level should be the “passing” mark for all. Test results are routinely reported as if those who did not meet this standard were “failing.”

I have routinely argued on this blog that NAEP proficiency is equivalent to earning an A, and that it was nuts to expect all students to earn an A. Only in one state (Massachusetts) have as many as 50% reached the standard.

Harvey demonstrates the reality.

He writes:

“In 1996, the International Education Assessment (IEA) released one of the earliest examinations of how well 4th grade students all over the world could read. IEA is a highly credible international institution that monitors comparative school performance; it also administers the Trends in Mathematics and Science Study (TIMSS), a global assessment of 4th and 8th grade mathematics and science achievement. Its 1996 assessment (The IEA Reading Literacy Study, a predecessor to the Progress in International Reading Literacy Study, or PIRLS) demonstrated that out of 27 participating nations, U.S. 4th graders ranked number two in reading (National Center for Education Statistics, 1996). Only Finland ranked higher. To the extent these rankings mean very much, this second-place finish for the United States was an impressive accomplishment.

”But around the same time, the National Assessment Governing Board of the National Assessment of Educational Progress (NAEP) reported that just one-third of American 4th graders were “proficient” in reading. To this day, the board of NAEP continues to release similarly bleak findings about American 4th graders’ reading performance (National Center for Education Statistics, 2011). And IEA continues to release global findings indicating that the performance of U.S. 4th graders in reading remains world class (Mullis et al., 2012).

“How could both these findings be accurate? Was it true, as NAEP results indicated, that U.S. 4th graders couldn’t walk and chew gum at the same time? Or was IEA’s conclusion—that the performance of American 4th graders in an international context was first class—more valid? A broader question arises here, one that has intrigued researchers for years: How would other nations perform if their students were held to the NAEP achievement-level benchmark for “proficient”? How might they perform on Common Core-aligned assess-ments with benchmarks that reflect those of NAEP?

”How Would Other Nations Score on NAEP?

“In 2015, statistician Emre Gönülates and I set out to explore these questions on behalf of the National Superintendents Roundtable (of which I am executive director) and the Horace Mann League (on whose board I serve). The results of our examination, recently released in a report titled How High the Bar? (Harvey & Gönülates, 2017), are eye-opening. In short, the vast majority of students in the vast majority of nations would not clear the NAEP bar for proficiency in reading, mathematics, or science. And the same is true of the “career and college-readiness” benchmarks in mathematics and English language arts that are used by the major Common Core-aligned assessments.

“This finding matters because in recent years, communities all over the United States have seen bleak headlines about the performance of their students and schools. Many of these headlines rely on reports about student achievement from NAEP or the Common Core assessments. One particular concern is that only a minority of students in the United States meet the NAEP Proficient benchmark. Frequently, arguments in favor of maintaining this particular benchmark as the desired goal for American students and education institutions are couched in terms of establishing demanding standards so the United States becomes more competitive internationally.

“But the reality is that communities around the world would face identical bleak headlines if their students sat down to take the NAEP assessments. So, when U.S. citizens read that “only one-third” or “less than half” of the students in their local schools are proficient in mathematics, science, or reading (or other subjects), they can rest assured that the same judgments could be applied to national education systems throughout the world if students in those nations participated in NAEP or Common Core-related assessments. (This is true despite the widespread perception that average student performance in some other nations exceeds average student performance in the United States. The metric applied in our study is not a rank ordering of mean scores by nation but the percentage of students in each nation likely to exceed the NAEP Proficient benchmark.)

“Our findings may not even be surprising when we consider questions that have arisen from previous research on NAEP.”

Harvey goes on to explain why it is absurd to use NAEP proficiency as a passing mark.

 

 

 

 

 

 

 

Categories Common Core, International, NAEP, Testing

Thomas Cox: In Praise of Our Public Schools

By dianeravitch
January 11, 2018 //
32

 

 

Thomas A. Cox practiced law in Georgia for many years and taught Education Policy and the Law at Emory University. Recently he moved to Virginia and discovered that the state’s leading newspaper, the Richmond Times Dispatch, was habitually hostile to the principle of public education and cheerleading for privatization. Cox submitted this opinion article to set the record straight, which the newspaper published.

I hope the editorial writers read his article.

He wrote about the falsity of the “failing schools narrative” and demonstrated that it is just plain wrong.

“Not often heard over the noise of this failure narrative is some compelling evidence that America’s public schools, far from being awash in failure, have overall been performing remarkably well, particularly in the face of new challenges and changing demographics. This counter-narrative is shared by a number of education researchers, historians, and educators, although they seldom receive the same fanfare (or financial impetus) as the nay-saying privatization advocates.”

Privatization is no answer to the challenges faced by our students today.

“A blind reliance on profit-driven markets to address and solve the challenges in educating America’s children would constitute a non-evidence-based leap of faith. Even worse, it would drive us toward abandoning our long-shared concept of education as a “common good” that we as a democratic polity have a collective responsibility to provide to all children. For almost two centuries, our country has served as a model to the world by striving to achieve that ideal through a shared societal commitment to publicly funded and locally operated schools.

“Although far from perfect and in need of constant re-evaluation and improvement, public schools and their legions of dedicated teachers continue to serve as critically important institutional forces in our nation’s ongoing struggle to provide equal opportunities for all citizens. In an age when so many economic and societal forces serve instead to increase inequality, now is no time for us to abandon that common commitment.”

 

Categories Accountability, Charter Schools, Corporate Reformers, Democracy, Education Industry, For-Profit, NAEP, NCLB (No Child Left Behind), Privatization, Supporting public schools

No, Patricia Levesque, Florida is Not A Model for the Nation

By dianeravitch
December 23, 2017 //
17

Patricia Levesque has worked for Jeb Bush for many years. She is his henchperson in promoting Florida as a miraculous story of educational improvement, based on Bush’s beliefs in high-stakes testing, test-based accountability, school report cards, and choice via charters, cybercharters, for-profit charters, and vouchers. The one belief he does not have is that public schools are important and valuable community institutions.

Here she is today, touting Florida as a “national model.” She says that Florida’s accountability system has “paid off” and is a roaring success.

The Bush approach may be briefly summarized as test-test-test, then close or privatize the schools that can’t produce the scores.

Let’s go to the videotape, or in this case, the NAEP scores for 2015.

In 2015, Florida scored at about the national average in 4th grade math but below the national average in 8th grade math.

Nineteen states had higher NAEP scores in 4th grade math than Florida.

Florida students in 8th grade math scored below the national average and were tied with their peers in South Carolina and Nevada.

Forty states had higher scores in 8th grade math than Florida in 2015.

In 4th grade reading in 2015, Florida was just above the national average.

Fifteen states recorded higher scores than Florida in 4th grade reading.

In 8th grade reading in 2015, Florida students were at the national average, tied with North Carolina and Georgia.

Thirty five states had higher reading scores on the NAEP in 2015 than students in Florida.

Why would anyone consider Florida to be a national model?

Why not choose a state like Massachusetts, which is #1 on all of these measures?

Who would choose to follow the practices of a state that scored at about the national average, rather than one of those states that consistently has greater success on the NAEP than Florida?

What Levesque fails to mention is that many states produced higher test scores over the past 20 years, and Florida’s relative position remained about the same. Florida has a policy of holding back third-graders based on test scores, so that probably inflates their 4th grade reading and math scores. The gains of Florida and other states may reflect that unrelenting emphasis on testing, pre-testing, interim testing, etc., which, as Daniel Koretz points out in his recent book “The Testing Charade” produces inflated scores.

If you live in Florida, check the facts before you follow the lead of Jeb Bush, who is trying to protect his “legacy” of high-stakes testing and privatization.

Bottom line: Florida is no national model, unless your goal is mediocrity.

Categories Accountability, Bush, Jeb, Education Reform, Lies, NAEP

Trump and Common Core: He Makes No Effort to Get Rid of It

By dianeravitch
October 13, 2017 //
41

It is not unusual to say that Trump lied about something. It happens every day.

But he does try to keep his campaign promises. He has tried and failed to build the Wall, and Mexico won’t pay for it. He has tried and failed to get rid of Obamacare.

But he hasn’t even tried to get rid of Common Core, which he promised to do. Everyone he interviewed for Education Secretary–including Eva Moskowitz and Michelle Rhee–supports Common Core.

Betsy DeVos was a supporter of Common Core before she became Secretary of Education, like her mentor Jeb Bush. She recently nominated at least three strong supporters of Common Core–former Michigan Governor John Engler, former North Carolina Governor Beverly Perdue, and test expert Greg Cizek, who helped develop one of the Common Core tests (Smarter Balanced Assessment)–to the governing board of the National Assessment of Educational Progress.

DeVos has not made any effort to discourage use of the Common Core.

Opponents of the CCSS: you were hoaxed! Trump will not get rid of it, nor will Betsy DeVos.

Categories Accountability, Betsy DeVos, Bush, Jeb, Common Core, Lies, NAEP

DeVos Appoints Former Governor Engler as Chair of NAEP Governing Board

By dianeravitch
September 29, 2017 //
11

Betsy DeVos apointed her friend and ally in the school choice movement, former Governor of Michigan John Engler, as chair of the National Assessment Governing Board, which administers the National Assessment of Educational Progress.

Engler is a charter member of the “schools-are-failing” club. He recently retired as president of the Business Roundtable, an association representing some of the nation’s leading businesses, and before that was head of the National Association of Manufacturers.

Expect every release of NAEP scores to be a dire warning about how terrible our public schools are, how we are no longer globally competitive, and why we need drastic steps (school choice?) to close the achievement gaps.

Maybe some enterprising journalist will ask Engler about the failure of his reforms in Michigan, as reflected in NAEP scores. Since Michigan became a laboratory for choice, its NAEP scores plummeted.

As a seven-year member of that board, I can tell you we zealously protected the scores from politicization. Don’t expect Engler to maintain that tradition.

Categories Betsy DeVos, Fraud, Michigan, NAEP

Study: NAEP Does Not Assess Critical Thinking Skills: No Standardized Test Can Do That

By dianeravitch
September 20, 2017 //
40

The link was left off. It is here.

Valerie Strauss reports on an important new study by a group at Stanford University led by historian Sam Wineburg.

NAEP supporters say that the tests are able to measure skills that other standardized tests can’t: problem solving, critical thinking, etc. But this post takes issue with that notion. It was written by three Stanford University academics who are part of the Stanford History Education Group: Sam Wineburg, Mark Smith and Joel Breakstone.

Wineburg, an education and history professor in the Graduate School of Education, is the founder and executive director of the Stanford History Education Group and Stanford’s PhD program in education history. His research interests include assessment, civic education and literacy. Smith, a former high school social studies in Iowa, Texas and California, is the group’s director of assessment; his research is focused on K-12 history assessment, particularly on issues of validity and generalizability. And Breakstone, a former high school history teacher in Vermont, directs the Stanford History Education Group. His research focuses on how teachers use assessment data to form instruction.

The National Assessment of Educational Progress is considered the “gold standard” of education testing because it is the only national longitudinal measure that goes back to 1970; no one can practice for it; no one knows which students will take the test; no single student takes the entire test; samples of students in every state take portions of the tests.

But when it comes to standardized testing, there is no gold standard. It is all dross, especially now that almost all standardized tests are delivered online. Online testing is popular because it is cheap and supposedly fast. But online testing by its nature allows no room for demonstrating thoughtfulness or for divergent thinking or for creative responses. It is the enemy of critical thinking.

Wineburg’s group tried to determine whether NAEP actually tested critical thinking, and they found that it did not.

But what would happen [they asked] if instead of grading the kids, we graded the test makers? How? By evaluating the claims they make about what their tests actually measure.

For example, in history, NAEP claims to test not only names and dates, but critical thinking — what it calls “Historical Analysis and Interpretation.” Such questions require students to “explain points of view,” “weigh and judge different views of the past,” and “develop sound generalizations and defend these generalizations with persuasive arguments.” In college, students demonstrate these skills by writing analytical essays in which they have to put facts into context. NAEP, however, claims it can measure such skills using traditional multiple-choice questions.

We wanted to test this claim. We administered a set of Historical Analysis and Interpretation questions from NAEP’s 2010 12th-grade exam to high school students who had passed the Advanced Placement (AP) exam in U.S. History (with a score of 3 or above). We tracked students’ thinking by having them verbalize their thoughts as they solved the questions.

What we learned shocked us.

In a study that appears in the forthcoming American Educational Research Journal, we show that in 108 cases (27 students answering four different items), there was not a single instance in which students’ thinking resembled anything close to “Historical Analysis and Interpretation.” Instead, drawing on canny test-taking strategies, students typically did an end run around historical content to arrive at their answers.

Their analysis is fascinating.

It is past time that we relinquished our obsession with standardized testing.

Categories History, NAEP, Standardized Testing

Cloaking Inequity: What NAEP Reveals About Charter Schools Vs. Public schools

By dianeravitch
May 19, 2017 //
40

Julian Vasquez Heilig, a scholar at California State University in Sacramento, reports on a research project comparing the performance of charter schools to public schools, using state scores on the National Assessment of Educational Progress (NAEP). NAEP is administered by the U.S. Department of Education and governed by a nonpartisan board appointed by the Secretary of Education. Since members serve for four-year terms, most were appointed by Arne Duncan or John King.

Heilig, with the assistance of Blake zclark, Jr., reviewed NAEP data and reached the following conclusions:

“One would most likely suspect from the current positive public discourse about charter schools that they would display higher national and large city NAEP performance when compared non-charter neighborhood schools, however, this is not actually the case when examining achievement data at the school level. Out of the 28 total comparison tests run, only 4 times did charters produce higher composite score averages than non-charter neighborhood public schools— 8th grade reading and math in the years 2013 and 2015. There was a tie in the large city comparison for 4th grade reading in the year 2013 as charter schools and non-charter neighborhood public schools displayed the same average composite scale scores. In the other 23 cases charter schools produced lower average composite scores on the NAEP (math, reading, science) than non-charter neighborhood public schools.”

The difference favoring public schools in 12th grade was very large.

In light of this disparity, why do so many federal and state policy makers consider charters a remedy for low-scoring public schools? What is the remedy for low-performing charter schools?

Categories Accountability, Charter Schools, NAEP, Privatization

Peter Goodman: How Would We Assess Students Without Standardized Tests?

By dianeravitch
April 30, 2017 //
26

Peter Goodman takes up the challenge that I put down a while back in a post about why we need standardized testing in every grade for every child.

It is worth noting that to my knowledge we are the only nation in the world that insists on testing every child from grade 3-8, and we have very little to show for it. Even if test scores went up, that wouldn’t mean that children are better educated. It means that we did a better job of test prep and teaching to the test. What happens to imagination and creativity when children are tested nonstop for years, given the instruction that every question has a right answer and only one right answer? None of us knows, but I doubt that it is good.

Peter notes that Regents tests have been around since the 1880s, but they were not required of every student until fairly recently, when New York Commissioner Rick Mills had the bright idea that no one should get a diploma unless he or she could pass five Regents exams. The exams were made simpler, to be sure; if the standards were kept high, most students would never finish high school.

Peter offers a number of examples of alternatives, all worth considering. The New York Performance Consortium does not administer the state exams, and their students do well in terms of high school graduation, college admission, and persistence in college.

Sometimes I wonder how my generation ever managed to acquire an education, since we almost never took standardized tests. The schools trusted our teachers to test us, using their own tests.

The only purpose of standardized tests is to compare students, to give them a ranking and a rating, but not to provide any information whatever about what they know and what they don’t know.

I said the standardized tests today are utterly useless because they provide no diagnostic information.

When my children were young, I never found out how they compared to other children. I got written reports from their teachers about their performance, where they were strong, and where they needed to work harder. I thought that was more than enough. Why are we so obsessed with comparing students in New York to students in other states? Do you care? If you do, there is NAEP, which gives you all the comparisons you need.

Categories NAEP, Standardized Testing

Gary Rubinstein: Was There a NAEP Miracle in Louisiana?

By dianeravitch
April 3, 2017 //
9

Gary Rubinstein has a somewhat startling habit of insisting on accuracy. He gets very annoyed when educators or pseudo-educators make claims that are false or only half-true or embellishments. I have worked with him on several occasions to track down the facts about “miracle schools” that turned out to be schools with high attrition rates or some other explanation of a dramatic spike in test scores or graduation rates.

In this post, he examines a claim made in an article by Louisiana Superintendent John White and Massachusetts Commissioner Mitchell Chester. Both of them are members of Jeb Bush’s Chiefs for Change, which is a strong indication that they are wedded to test scores and school choice.

Chester comes from a state that has historically been the highest-performing in the nation.

What bothers Rubinstein is that White uses the article to claim some sort of Louisiana “miracle” on his watch, and he cites NAEP scores. That sets off alarm bells for Rubinstein.

This is White’s claim:

In Louisiana, radical change means that 128,000 fewer students attend schools rated D or F than did in 2011. That’s had a powerful impact on the historically disadvantaged children too often consigned to failing schools, vaulting the performance of African-American fourth graders into the middle of the pack on the National Assessment of Educational Progress in 2015. In 2009, for example, black fourth graders ranked 43rd and 41st in the nation for proficiency in reading and math, respectively. Those rankings jumped to 20th and 23rd in 2015.

Rubinstein writes:

As far as the 128,000 fewer students attending schools rated D or F, since they are the ones who assign those ratings and since the criteria for getting a D or F has changed over the years, I don’t take that one too seriously.

But I was interested in ‘fact checking’ that NAEP statistic since that was one I hadn’t heard of before. I knew that Louisiana as a whole had very low NAEP scores and they were not improving very much over the years the way, for example Tennessee and Washington D.C. have, otherwise we’d be hearing about Louisiana NAEP much more.

White says that black fourth graders ranked 43rd in reading and 41st in math in 2009 and now rank 20th and 23rd. So I went to the National Center for Education Statistics website and dug into the data.

Since NAEP isn’t just for 4th graders, the first thing I checked was what their current ranking was for black 8th graders and saw that for 8th grade math they actually dropped from 39th to 44th between 2009 and 2015. For 8th grade reading they dropped from to 43rd to 45th between 2009 and 2015. So it is obvious why they don’t mention their 8th grade change in rankings.

I also checked how they have done in math for all 4th graders regardless of race. I found that in 2009 they were 48th while in 2015 they were not much better, at 44th. In reading they went from second to last in 2009 to 8th to last in 2015. A jump, but not the sort of thing that John White would ever use to prove his point about his knowledge of improving schools.

But he went on to inquire about the statistical significance of the fourth grade gains.

What he learned will surprise you.

Categories Louisiana, Massachusetts, NAEP

New Study: Michigan Test Score Gains are Lowest in the Nation

By dianeravitch
February 21, 2017 //
16

Will Betsy DeVos do to the nation what she has done to Michigan?

A new study by Professor Brian Jacob of the University of Michigan demonstrates that Michigan’s gains on the National Assessment of Educational Progress were the lowest in the nation. Now we know why DeVos was unable to explain the difference between “growth” and “proficiency” at her Senate hearings. She really has no idea that her own state has been stagnant as her philosophy of choice took hold. Brookings has not yet posted the study online. But the Detroit News reported its results and interviewed Jacob about his findings.

A new analysis of results of a national educational test shows Michigan students have continually made the least improvement nationally of scores since 2003.

The study, by University of Michigan professor Brian A. Jacob, of scores of the National Assessment of Education Progress (NAEP), also found that Michigan students were at the bottom of the list when it comes to proficiency growth in the four measures of the exam.

That analysis comes less than six months after the release of the Michigan’s Talent Crisis report by Education Trust-Midwest that found Michigan’s students are falling far behind their peers across the nation. The ETM report found that Michigan is in the bottom 10 states for key subjects and grades, including early literacy.

Fourth-grade scores have been stagnant. In eighth-grade, Michigan has not kept pace with gains made by other states.

Jacob, a nonresident senior fellow at the Brookings Institute, said there is no single explanation of the Michigan rankings.

“I believe that there are a number of factors responsible for Michigan’s weak performance: a lack of adequate state and local funding for schools, the highly decentralized nature of governance that makes it difficult for the state Department of Education to develop coordinated reforms, the lack of regulation and accountability in the charter sector, and the economic and political instability that have plagued Detroit and other urban areas in the state,” he said.

“Another reason is the relatively decentralized nature of education in Michigan,” he added. “The long tradition of local control in the state has made it harder for the state Department of Education or others to establish coordinated policies.”

Jacob said those factors particularly affect Detroit. “The political and financial instability of Detroit over the past decade or two undoubtedly had a major impact on student performance in the city and surrounding areas,” he said.

As expected, Republican officeholders said that throwing money at the problem won’t help. Obviously, charters don’t help either, since they are flourishing and unaccountable and not producing better results than public schools.

Categories Accountability, Betsy DeVos, Charter Schools, Michigan, NAEP
« Older Entries
Newer Entries »