Archives for category: NAEP
« Older Entries
Newer Entries »

Peter Goodman: How Would We Assess Students Without Standardized Tests?

By dianeravitch
April 30, 2017 //
26

Peter Goodman takes up the challenge that I put down a while back in a post about why we need standardized testing in every grade for every child.

It is worth noting that to my knowledge we are the only nation in the world that insists on testing every child from grade 3-8, and we have very little to show for it. Even if test scores went up, that wouldn’t mean that children are better educated. It means that we did a better job of test prep and teaching to the test. What happens to imagination and creativity when children are tested nonstop for years, given the instruction that every question has a right answer and only one right answer? None of us knows, but I doubt that it is good.

Peter notes that Regents tests have been around since the 1880s, but they were not required of every student until fairly recently, when New York Commissioner Rick Mills had the bright idea that no one should get a diploma unless he or she could pass five Regents exams. The exams were made simpler, to be sure; if the standards were kept high, most students would never finish high school.

Peter offers a number of examples of alternatives, all worth considering. The New York Performance Consortium does not administer the state exams, and their students do well in terms of high school graduation, college admission, and persistence in college.

Sometimes I wonder how my generation ever managed to acquire an education, since we almost never took standardized tests. The schools trusted our teachers to test us, using their own tests.

The only purpose of standardized tests is to compare students, to give them a ranking and a rating, but not to provide any information whatever about what they know and what they don’t know.

I said the standardized tests today are utterly useless because they provide no diagnostic information.

When my children were young, I never found out how they compared to other children. I got written reports from their teachers about their performance, where they were strong, and where they needed to work harder. I thought that was more than enough. Why are we so obsessed with comparing students in New York to students in other states? Do you care? If you do, there is NAEP, which gives you all the comparisons you need.

Categories NAEP, Standardized Testing

Gary Rubinstein: Was There a NAEP Miracle in Louisiana?

By dianeravitch
April 3, 2017 //
9

Gary Rubinstein has a somewhat startling habit of insisting on accuracy. He gets very annoyed when educators or pseudo-educators make claims that are false or only half-true or embellishments. I have worked with him on several occasions to track down the facts about “miracle schools” that turned out to be schools with high attrition rates or some other explanation of a dramatic spike in test scores or graduation rates.

In this post, he examines a claim made in an article by Louisiana Superintendent John White and Massachusetts Commissioner Mitchell Chester. Both of them are members of Jeb Bush’s Chiefs for Change, which is a strong indication that they are wedded to test scores and school choice.

Chester comes from a state that has historically been the highest-performing in the nation.

What bothers Rubinstein is that White uses the article to claim some sort of Louisiana “miracle” on his watch, and he cites NAEP scores. That sets off alarm bells for Rubinstein.

This is White’s claim:

In Louisiana, radical change means that 128,000 fewer students attend schools rated D or F than did in 2011. That’s had a powerful impact on the historically disadvantaged children too often consigned to failing schools, vaulting the performance of African-American fourth graders into the middle of the pack on the National Assessment of Educational Progress in 2015. In 2009, for example, black fourth graders ranked 43rd and 41st in the nation for proficiency in reading and math, respectively. Those rankings jumped to 20th and 23rd in 2015.

Rubinstein writes:

As far as the 128,000 fewer students attending schools rated D or F, since they are the ones who assign those ratings and since the criteria for getting a D or F has changed over the years, I don’t take that one too seriously.

But I was interested in ‘fact checking’ that NAEP statistic since that was one I hadn’t heard of before. I knew that Louisiana as a whole had very low NAEP scores and they were not improving very much over the years the way, for example Tennessee and Washington D.C. have, otherwise we’d be hearing about Louisiana NAEP much more.

White says that black fourth graders ranked 43rd in reading and 41st in math in 2009 and now rank 20th and 23rd. So I went to the National Center for Education Statistics website and dug into the data.

Since NAEP isn’t just for 4th graders, the first thing I checked was what their current ranking was for black 8th graders and saw that for 8th grade math they actually dropped from 39th to 44th between 2009 and 2015. For 8th grade reading they dropped from to 43rd to 45th between 2009 and 2015. So it is obvious why they don’t mention their 8th grade change in rankings.

I also checked how they have done in math for all 4th graders regardless of race. I found that in 2009 they were 48th while in 2015 they were not much better, at 44th. In reading they went from second to last in 2009 to 8th to last in 2015. A jump, but not the sort of thing that John White would ever use to prove his point about his knowledge of improving schools.

But he went on to inquire about the statistical significance of the fourth grade gains.

What he learned will surprise you.

Categories Louisiana, Massachusetts, NAEP

New Study: Michigan Test Score Gains are Lowest in the Nation

By dianeravitch
February 21, 2017 //
16

Will Betsy DeVos do to the nation what she has done to Michigan?

A new study by Professor Brian Jacob of the University of Michigan demonstrates that Michigan’s gains on the National Assessment of Educational Progress were the lowest in the nation. Now we know why DeVos was unable to explain the difference between “growth” and “proficiency” at her Senate hearings. She really has no idea that her own state has been stagnant as her philosophy of choice took hold. Brookings has not yet posted the study online. But the Detroit News reported its results and interviewed Jacob about his findings.

A new analysis of results of a national educational test shows Michigan students have continually made the least improvement nationally of scores since 2003.

The study, by University of Michigan professor Brian A. Jacob, of scores of the National Assessment of Education Progress (NAEP), also found that Michigan students were at the bottom of the list when it comes to proficiency growth in the four measures of the exam.

That analysis comes less than six months after the release of the Michigan’s Talent Crisis report by Education Trust-Midwest that found Michigan’s students are falling far behind their peers across the nation. The ETM report found that Michigan is in the bottom 10 states for key subjects and grades, including early literacy.

Fourth-grade scores have been stagnant. In eighth-grade, Michigan has not kept pace with gains made by other states.

Jacob, a nonresident senior fellow at the Brookings Institute, said there is no single explanation of the Michigan rankings.

“I believe that there are a number of factors responsible for Michigan’s weak performance: a lack of adequate state and local funding for schools, the highly decentralized nature of governance that makes it difficult for the state Department of Education to develop coordinated reforms, the lack of regulation and accountability in the charter sector, and the economic and political instability that have plagued Detroit and other urban areas in the state,” he said.

“Another reason is the relatively decentralized nature of education in Michigan,” he added. “The long tradition of local control in the state has made it harder for the state Department of Education or others to establish coordinated policies.”

Jacob said those factors particularly affect Detroit. “The political and financial instability of Detroit over the past decade or two undoubtedly had a major impact on student performance in the city and surrounding areas,” he said.

As expected, Republican officeholders said that throwing money at the problem won’t help. Obviously, charters don’t help either, since they are flourishing and unaccountable and not producing better results than public schools.

Categories Accountability, Betsy DeVos, Charter Schools, Michigan, NAEP

Michigan, Under DeVos Influence, Declines Sharply on NAEP

By dianeravitch
February 19, 2017 //
30

The Detroit Free Press published this article last May about the state’s academic decline since 2003. That covers the DeVos era.

http://www.freep.com/story/news/education/2016/05/18/michigan-students-sliding-toward-bottom/84535876/

“In 2003, Michigan ranked 28th in fourth-grade reading. In 2015, the state was ranked 41st.
“We’re certainly not on track to become a top 10 state any time soon,” said Amber Arellano, executive director of the organization. “It’s totally unacceptable for the economy, for business and especially for kids themselves.”

Among the 2015 NAEP results highlighted in the report:

• Michigan ranked 41st in fourth-grade reading, down from 28th in 2003.

• The state ranked 42nd in fourth-grade math, down from 27 in 2003.

• It ranked 31st in eighth-grade reading, down from 27th in 2003.

• It ranked 38th in eight-grade math, down from 34th.

The report is focused on the fourth-grade reading results because of how crucial it is for students to be able to read well by the end of third grade. But students have also struggled in math.

The achievement problem crosses demographic lines. Consider how various demographic groups in Michigan compared with similar demographic groups nationwide in fourth-grade reading in 2015: White students in Michigan ranked 49th, higher-income students in Michigan ranked 48th, and black students ranked 41st.

The problem? Many other states are outpacing Michigan, which has posted mostly stagnant — and in some cases declining — results.”

Will DeVos inflict her failed methods on the nation?

Categories Betsy DeVos, Michigan, NAEP, Privatization, School Choice

Michigan: Academic Performance Has Been Declining as “Choice” Increases

By dianeravitch
January 17, 2017 //
8

 

Spurred by the financial clout and political power of the DeVos family, Michigan has embraced choice. A charter advocate wrote earlier to claim that the state has made unparalleled gains, thanks to choice. I knew this was wrong, but was on a car trip and couldn’t look up the NAEP data. In fact, Michigan’s academic performance relative to other states is in free fall.

 

On the National Assessment of Educational Progress, the only reliable barometer of test performance, Michigan has gone into a decline over the past dozen years.

 

Michigan, already sliding toward the bottom nationally for fourth-grade reading performance on a rigorous national exam, is projected to fall to 48th place by 2030 if the state does nothing to improve education.

 

That finding is included in a report out today from Education Trust-Midwest, a nonpartisan education research and policy organization based in Royal Oak. The organization analyzed more than a decade’s worth of results from the National Assessment of Educational Progress — or NAEP, a tough exam given to a representative sample of students in each state.

 

In 2003, Michigan ranked 28th in fourth-grade reading. In 2015, the state was ranked 41st.

 

“We’re certainly not on track to become a top 10 state any time soon,” said Amber Arellano, executive director of the organization. “It’s totally unacceptable for the economy, for business and especially for kids themselves.”
Among the 2015 NAEP results highlighted in the report:

 

• Michigan ranked 41st in fourth-grade reading, down from 28th in 2003.

 

• The state ranked 42nd in fourth-grade math, down from 27 in 2003.

 

• It ranked 31st in eighth-grade reading, down from 27th in 2003.

 

• It ranked 38th in eight-grade math, down from 34th.

 

The report is focused on the fourth-grade reading results because of how crucial it is for students to be able to read well by the end of third grade. But students have also struggled in math.

 

The achievement problem crosses demographic lines. Consider how various demographic groups in Michigan compared with similar demographic groups nationwide in fourth-grade reading in 2015: White students in Michigan ranked 49th, higher-income students in Michigan ranked 48th, and black students ranked 41st.

 

The problem? Many other states are outpacing Michigan, which has posted mostly stagnant — and in some cases declining — results on the NAEP.

 

“When you look at leading states … they’re like on a rocket ship and we’re on a snail,” Arellano said.

 

State officials are busily mapping plans and goals to become one of the top 10 states in the nation. But they are falling farther and farther down towards the bottom. If they keep up the DeVos formula, they will soon rank among the Southern states, where academic achievement has historically been low because of underfunding and high poverty.

 

Detroit has most of the charter schools in the state of Michigan. It is the lowest ranking urban district on the National Assessment of Education Progress. Many of the charter schools are far worse academically than the chronically underfunded public schools.

 

Don’t let anyone tell you that Michigan or Detroit have been improved by choice. The only reliable measure is the NAEP, and both Michigan and the city of Detroit are in terrible shape.

 

 

 

Categories Accountability, Charter Schools, Detroit, Failure, Michigan, NAEP, Privatization, School Choice

A Misinformed, Embarrassing Article from Brookings about Our “Failing Schools”

By dianeravitch
January 4, 2017 //
18

The Brookings Institution was once known as a reliable source of thoughtful, informed analysis of important policy issues. In the past decade, it has turned its education commentary over to rightwing ideologues, who are driven by ideology and indifferent to facts that they ought to know.

 

On behalf of Brookings, Jonathan Rothwell, economist for Gallup, complains that the U.S. spends more on education, has seen no improvement in decades, and is seeing no gains in productivity. He ends by saying that low-income families can’t afford private tutors or home schooling, as though these were viable ways to improve education for the poorest children .

 

I can’t unpack all this in a short space, but I would like to show you in a few paragraphs why this is an uninformed article. To begin with, Rothwell cherrypicks the data on test scores. This makes his analysis misleading and wrong. Test scores are the highest they have ever been on the only longitudinal measure we have: the National Assessment of Educational Progress (NAEP). He selectively quotes one version of the NAEP, while ignoring the other.

 

There are actually two versions of NAEP. One is called the “Long Term Trend” (LTT) data, the other is main NAEP. The LTT is offered every four years to samples of students at age 9, 13, and 17. Main NAEP is given every other year to students in grades 4 and 8.

 

LTT contains questions that are unchanged since the early 1970s and have no relation to what students are taught today. Occasionally, questions are deleted because their content is obsolete (e.g., a question that refers to S&H Green Stamps). The data for 17-year-olds is especially dubious because this group has no incentive to take the NAEP tests seriously.

 

The National Assessment Governing Board is aware of the problem of low motivation among 17-year-olds. When I served on the board, from 1994-2001, we devoted a large part of one of our quarterly meetings to this problem. There was talk of incentives, pizza parties, cash, but it was not resolved. The bottom line, however, is that any data about the test scores of 17-year-olds must acknowledge that this group doesn’t care about the test because they know it doesn’t matter. What the board learned when we discussed it is that some 17-year-olds doodle on the answer sheet or answer every question by checking the same letter. They don’t care.

 

I recommend that Rothwell read Chapter 5 of my book Reign of Error. He would learn there that the scores on the main NAEP reached their highest point ever in 2013 (they were flat for the first time in many years in 2015). This was true for every group of whites, blacks, Hispanics, and Asians. He would also learn that the graduation rate was the highest ever for these groups, and the dropout rate was the lowest ever. He would see a different reading of the LTT data, showing a dramatic rise in test scores in math for black students and Hispanic students in all three age groups, and for white students at ages 9 and 13, from 1973 to 2008. Even white 17-year-olds saw a gain, but it was small.

 

If I may quote my analysis, based on a review of both versions of NAEP, “NAEP data show beyond question that test scores in reading and math have improved for almost every group of students over the past two decades: slowly and steadily in the case of reading, dramatically in the case of mathematics.”

 

I would also urge Rothwell to read chapter 7, which reviews the international test scores. It shows that we were never #1 in test scores on international tests. In fact, when the first international tests were given in 1964, we were last among 12 nations. Yet over the half century that followed, we outpaced all the other 11 nations by every measure.

 

I know that Brookings uses Google or some other search engine to find anything that quotes its articles and research. I hope that they find this article and bring it to the attention of Jonathan Rothwell.

 

More important, I can only hope and wish that Brookings would make the effort to employ genuine education researchers to write and declaim about this important subject. Over the past decade, its education spokesman was Grover Whitehurst, George W. Bush’s former research director, who turned Brookings into a cheerleading think tank for school choice. This is unworthy of a once-great and once-trusted institution.

 

Categories Accountability, International, NAEP, Reign of Error, Testing

Politico: Michigan is a Horrible Model for U.S. Education

By dianeravitch
December 9, 2016 //
19

Caitlin Emma, Benjamin Wermund, and Kimberly Hefling, staff writers at politico.com, took a close look at Michigan and answered the question, what hath Betsy DeVos’s obsession with choice done to the schools of Michigan?

 

Unless you are a choice fanatic like DeVos, the answer is not encouraging.

 

Despite two decades of charter-school growth, the state’s overall academic progress has failed to keep pace with other states: Michigan ranks near the bottom for fourth- and eighth-grade math and fourth-grade reading on a nationally representative test, nicknamed the “Nation’s Report Card.” Notably, the state’s charter schools scored worse on that test than their traditional public-school counterparts, according to an analysis of federal data.

 

Critics say Michigan’s laissez-faire attitude about charter-school regulation has led to marginal and, in some cases, terrible schools in the state’s poorest communities as part of a system dominated by for-profit operators. Charter-school growth has also weakened the finances and enrollment of traditional public-school districts like Detroit’s, at a time when many communities are still recovering from the economic downturn that hit Michigan’s auto industry particularly hard.

 

The results in Michigan are so disappointing that even some supporters of school choice are critical of the state’s policies.

 

“The bottom line should be, ‘Are kids achieving better or worse because of this expansion of choice?’” said Michigan State Board of Education President John Austin, a DeVos critic who also describes himself as a strong charter-school supporter. “It’s destroying learning outcomes … and the DeVoses were a principal agent of that.”

 

The links are in the article, as well as a puzzle. Check out the link to CREDO at Stanford (funded by the Walton Foundation), which issued a report on Michigan charters and praised them extensively. How does the CREDO finding make sense to Michigan’s low standing on the National Assessment of Education Progress? How does it make sense in light of the fact that Detroit is the worst-performing urban district tested by NAEP?

 

 

Categories Accountability, Betsy DeVos, Charter Schools, Detroit, Education Industry, Michigan, NAEP, Privatization, Testing

Just When You Thought That “Grit” Had Been Discredited, Here Come the Federal Grit Tests

By dianeravitch
July 1, 2016 //
32

The federal test, the National Assessment of Education Progress–called “The Nation’s Report Card”–will begin testing “grit.” Grrr.

Politico.com reports:

QUESTIONING GRIT: How do issues such as “grit” – or perseverance – and school climate correlate to student test performance? Following next year’s National Assessment of Educational Progress, researchers hope to have more clues. The student questionnaire on NAEP in 2017 will include a series of questions that address what are known as “non-cognitive” skills. By asking several questions within the topic areas, acting NCES Commissioner Peggy Carr tells Morning Education they will get “multiple chances to get the best measurement possible.” Researchers will take the answers to create an index that’s compared with test scores to create a composite designed to reflect the relationship between factors such as grit and academic achievement. Students will also be asked about technology use and their socioeconomic status.

– A glance at questions piloted earlier this year suggests what researchers are looking for [http://1.usa.gov/296dnLY ], although how the questionnaire will look next year could change. One pilot question, for example, asked fourth graders if they have a “problem while working toward an important goal, how well can you keep working?” Another question asks how often they “felt left out of things at school?” The questionnaire typically takes students about 15 minutes to fill out, and is optional.

– These types of questions have long helped researchers and policymakers to better understand student learning, said Bill Bushaw, the executive director of the National Assessment Governing Board. He said no personally-identifying information is collected from the sample of students taking NAEP, and no questions are asked about personal beliefs or religion. “It really helps understand the differences in academic achievement and that’s really what this is about,” Bushaw said. For example, in the past, Bushaw said a question that asked students how many times they’d been absent in the last month led researchers to conclude that students who miss a lot of school have lower achievement. School districts have responded by focusing more on school attendance.

Just think. We will soon know which state has the grittiest students. Which race and ethnicity and gender needs more grit. Where the grit gap is. The possibilities for research are amazing!

Categories Grit, NAEP

Tom Loveless: Why Campbell Brown, StudentsFirst, and Achieve Are Wrong about NAEP Proficient

By dianeravitch
June 14, 2016 //
10

Tom Loveless of the Brookings Institution has studied student achievement for many years. He has written several reports on the National Assessment of Educational Progress (NAEP). Before earning his doctorate, he taught sixth grade in California.

In this post, he explains why “reformers” who confuse NAEP’s “proficient level” with “grade level” are wrong.

This claim has been asserted by pundits like Campbell Brown of The 74, Michelle Rhee, and organizations such as Achieve. They want the public to believe that our public schools are failing miserably, and our kids are woefully dumb. But Loveless shows why they are wrong.

He writes:

Equating NAEP proficiency with grade level is bogus. Indeed, the validity of the achievement levels themselves is questionable. They immediately came under fire in reviews by the U.S. Government Accountability Office, the National Academy of Sciences, and the National Academy of Education.[1] The National Academy of Sciences report was particularly scathing, labeling NAEP’s achievement levels as “fundamentally flawed.”

Despite warnings of NAEP authorities and critical reviews from scholars, some commentators, typically from advocacy groups, continue to confound NAEP proficient with grade level. Organizations that support school reform, such as Achieve Inc. and Students First, prominently misuse the term on their websites. Achieve presses states to adopt cut points aligned with NAEP proficient as part of new Common Core-based accountability systems. Achieve argues that this will inform parents whether children “can do grade level work.” No, it will not. That claim is misleading.

The expectation that all students might one day reach 100% proficiency on the NAEP is completely unrealistic. It has not happened in any other country, including the highest performing. Not even our very top students taught by our very best teachers haven’t reached 100% proficiency. This is a myth that should be discarded.

Loveless goes even farther and insists that NAEP achievement levels should not be the benchmark for student progress.

He warns:

Confounding NAEP proficient with grade-level is uninformed. Designating NAEP proficient as the achievement benchmark for accountability systems is certainly not cautious use. If high school students are required to meet NAEP proficient to graduate from high school, large numbers will fail. If middle and elementary school students are forced to repeat grades because they fall short of a standard anchored to NAEP proficient, vast numbers will repeat grades.

Anyone who claims that NAEP proficient is the same as grade level should not be taken seriously. Loveless doesn’t point out that the designers of the Common Core tests decided to align their “passing mark” with NAEP proficient, which explains why 70% of students typically fails the PARCC and the SBAC tests. Bear in mind that the passing mark (the cut score) can be arbitrarily set anywhere–so that all the students “pass,” no students pass, or some set percentage will pass. That’s because the questions have been pre-tested, and test developers know their level of difficulty. And that is why U.S. Secretary of Education John King, when he was New York Commissioner of Education, predicted that only 30% of the students who took the state tests would “pass.” He was uncannily accurate because he already knew that the test was designed to “fail” 70%.

He concludes:

NAEP proficient is not synonymous with grade level. NAEP officials urge that proficient not be interpreted as reflecting grade level work. It is a standard set much higher than that. Scholarly panels have reviewed the NAEP achievement standards and found them flawed. The highest scoring nations of the world would appear to be mediocre or poor performers if judged by the NAEP proficient standard. Even large numbers of U.S. calculus students fall short.

As states consider building benchmarks for student performance into accountability systems, they should not use NAEP proficient—or any standard aligned with NAEP proficient—as a benchmark. It is an unreasonable expectation, one that ill serves America’s students, parents, and teachers–and the effort to improve America’s schools.

Categories Corporate Reformers, Hoax, NAEP, Testing

Mercedes Schneider Tries to Explain NAEP to Campbell Brown

By dianeravitch
May 24, 2016 //
20

Mercedes Scheider followed the contretemps around Campbell Brown’s misuse of NAEP data. She decided she would try to educate Campbell.

 

Mercedes goes through NAEP scores over many years and shows how they went up nicely until 2015, when they stagnated at about 1/3 proficient. She patiently explains that until 2015, scores were going up, not declining as Campbell believes.

 

Now for Campbell to understand this, she is going to have to read it and think about it.

 

Will she?

 

Will she correct her error about 2/3 of US kids in 8th grade being “below grade level”? It is not true.

Categories Education Reform, NAEP
« Older Entries
Newer Entries »