Archives for category: NAEP
« Older Entries
Newer Entries »

Carol Burris Tries to Correct Campbell Brown, Without Success

By dianeravitch
May 24, 2016 //
26

Carol Burris wrote a post for Valerie Strauss’s “Answer Sheet” blog at the Washington Post, in which she reported the numerous Twitter exchanges among herself, Tom Loveless, and Campbell Brown. Burris and Loveless fruitlessly tried to persuade Brown to retract her inaccurate statement that 2/3 of America’s eighth graders score below grade level.

 

Given an opportunity to respond by Valerie Strauss, Brown made an incomprehensible statement about how she should have referred to grade-level proficiency instead of grade level. Then everything would be okay. Instead of correcting her error, Brown insisted she was under personal attack.

 

Please read the last two sentences of her comments, which are hilarious. Especially the reference to “the age of Donald Trump and Diane Ravitch” (Sic)!! And then there is her laughable claim that those who disagree with her negative comments are profiting from school failure. I wish she–who received $4 million to start her website–would provide evidence for that statement!

 

 

Categories Accountability, Corporate Reform, Education Reform, NAEP, Testing

Education Expert Campbell Brown Offers Advice to the Next President

By dianeravitch
May 18, 2016 //
80

The rheeform leadership has changed. Michelle Rhee was once the cover girl for test-and-punish reform, and now it is Campbell Brown. The telegenic Brown used to read the news on television but now she has taken Rhee’s place in the reformy firmament. Since she launched her career as an education expert with an op-ed attacking the teachers’ union in New York City for protecting sexual predators, Brown has become increasingly active in the world of education punditry. She received $4 million from various billionaires to launch a news site called “The 74,” which was supposed to refer to the number of school-age children in the United States. However, there are 50 million school-age children, but then why quibble? Brown organized candidate debates for both parties last fall. Three Republicans showed up, and no Democrats. Yesterday, she moderated a panel at the Harvard Graduate School of Education at a symposium on poverty and schooling.

 

Now Brown, having established her bona fides as an expert on education, has prepared a memo for the next president. 

 

Unfortunately her memo begins with a false statement. She starts by saying that 2/3 of American students in eighth grade are “below grade level” in reading and math. Apparently she refers to the National Assessment of Education Progress, the only national assessment of student skills. She confuses NAEP proficiency, a specific achievement level, with grade level.

 

To begin with, “grade level” is a median. Fifty percent are always above grade level, and fifty percent are always below.

 

But the NAEP achievement levels do not measure “grade level.” They are defined in the NAEP reports thus: “basic” represents partial mastery of skills; “proficiency” represents mastery; “advanced” represents extraordinary performance. “Below basic” is very poor performance.

 

Here are the definitions on the NAEP website:

 

 

Achievement Level Policy Definitions
Basic

 

Partial mastery of prerequisite knowledge and skills that are fundamental for proficient work at each grade.
Proficient

Solid academic performance for each grade assessed. Students reaching this level have demonstrated competency over challenging subject matter, including subject-matter knowledge, application of such knowledge to real-world situations, and analytical skills appropriate to the subject matter.
Advanced

 

Superior performance.

 

Here is a statement on the U.S. Department of Education (National Center for Education Statistics) website:

 

https://nces.ed.gov/nationsreportcard/studies/statemapping/faq.aspx

The statement, “Proficient is not synonymous with grade-level performance.”

 

The NAEP website says that the Governing Board thinks that the goal should be “proficient,” not “basic,” but the reality is that these achievement levels have been in place since 1992, and in no state or district has 100% of students ever achieved NAEP proficiency. In only one state, Massachusetts, has as much as 50% of students reached proficiency. If you believe, as Campbell Brown and the NAGB board does, that 100% of students should reach proficiency, then you believe that somewhere there is a baseball team that never loses a game, or an entire school district in which all children get grades of all As. It has never happened, not even in the wealthiest, most successful schools and districts. When elephants can fly, that is when “all” students will reach NAEP proficiency. Be it noted that the standard (the passing mark or cut score) for the Common Core tests is aligned with NAEP proficient, which is why 65-70% of students consistently “fail.”

 

I was a member of the National Assessment Governing Board for seven years. I read questions before they were administered to samples of students across the nation and in every state. The greatest number of students are scored as “basic,” which I consider to be the equivalent of a B or C. Those who register as “proficient” are the equivalent of an A performance. Advanced is for superstars. Typically, only 5-10% of students are “advanced.” About a third are proficient or advanced. The remaining 65% are basic or below basic. (These are my definitions, not the government’s or the NAGB board.)

 

To expect that most students will score the equivalent of an A is nonsensical.

 

Ms. Brown has been engaged in a Twitter debate with Tom Loveless of the Brookings Institution. Loveless, a real expert with years of teaching experience (elementary school) and a doctorate, has been studying and writing about NAEP and student performance for many years. He chastised Brown on Twitter for saying that 2/3 of students are “below grade level.” He encouraged her to check her facts, because he assumed that her journalistic background had taught her to do so. Carol Burris, director of the Network for Public Education, and veteran educator, jumped into the exchange.

 

Just yesterday, Brown responded with this comment:

 

Campbell Brown ‏@campbell_brown 
@carolburris @tomloveless99 this is why parents dont listen to u. U play semantics while 2/3 kids arent where they should be. I call BS

 

Since Brown thinks that NAEP proficiency is the same as “grade level,” she would profit by reading this report on the meaning of NAEP achievement levels. It gives a good overview of them and points out that they do not refer to grade levels. The report also usefully reviews the numerous critiques of the achievement levels, by experts who consider them “fundamentally flawed” and an inaccurate measure of student achievement.

 

I can only hope that Ms. Brown, education expert, gets a quick tutorial about what NAEP achievement levels are.

 

And I invite her to take the NAEP eighth-grade test, composed of released questions in reading and math, and release her scores. In a supervised setting, of course. I think she will be surprised. I will be interested to see if she is “proficient,” since she believes that anyone who is not proficient is a failure.

 

 

Categories Accountability, Corporate Reformers, Education Reform, NAEP, Testing

Low-Performing Students Decline Most on NAEP Test of Seniors

By dianeravitch
April 29, 2016 //
31

The recent release of the test scores of seniors on the National Assessment of Educational Progress reported that low-performing students suffered the biggest declines.

“Much like their 4th and 8th grade peers, high school seniors have lost ground in math over the last two years, according to the most recent scores on a national achievement test.

“In reading, 12th grade scores remained flat, continuing a trend since 2009.

“Perhaps the most striking detail in the test data, though, is that the lowest achievers showed large score drops in both math and reading. Between 2013 and 2015, students at or below the 10th percentile in reading went down an average of 6 points on the National Assessment for Educational Progress—the largest drop in a two-year period since 1994. The high achievers, on the other hand—those at or above the 90th percentile—did significantly better in reading, gaining two points, on average, while staying stagnant in math.”

Categories NAEP

John Thompson: A Closer Look at the “Success” in D.C. That Wasn’t

By dianeravitch
December 15, 2015 //
8

John Thompson knows that reformers point to the District of Columbia as one of their examples of success. After all, the district has been controlled by Teach for America alumnae Michelle Rhee and Kaya Henderson since 2007. They own whatever successes and not-successes that occurred over the past eight years. The centerpiece of their claims of success is NAEP scores, which are up.

 

In this post, Thompson identifies the flaws in the narrative of success. Thompson lauds John Merrow for critiquing the narrative of a district he once held up as an exemplar of successful reform. Merrow asked, in his post, why anyone was celebrating Kaya Henderson’s five-year anniversary in the wake of the disastrous scores on the Common Core PARCC tests, which showed a district where academic performance was dismal.

 

Thompson reviews the NAEP scores, using Rick Hess’s data.

 

Hess cites overall gains in NAEP growth under Rhee and Henderson, but those same NAEP studies actually support the common sense conclusion that the numbers reflect gentrification. Hess’s charts show that from 2005 to 2013, the percentage of D.C. students who are low-income dropped from 66% to 61.6%. (In my world, a 61.6% low-income urban school seems danged-near rich.) Per student spending increased by 40% during that time. (The new spending, alone, comes close to the total per student spending in my 90% low-income system.)

 

According to Hess’s chart, the percentage of the D.C. students who are black dropped by 1/8th from 2005 to 2013, and the percentage of students with disabilities dropped by 1/7th. And, the 2015 NAEP excluded as many as 44% of D.C.’s English Language Learners. The conservative reformer RiShawn Biddle calls that exclusion “massive and unacceptable test-cheating.”

 

Even so, as Merrow reminds us, the performance gap between low-income and more affluent students has grown even wider; for instance, from 2002 to 2015, the 8th grade reading performance gap grew from 17 to 48 points.

 

Before Rhee/Henderson, the growth in D.C. test scores was spread much more widely. Because I believe that 8th grade reading is the most important NAEP metric in terms of evaluating school performance, I will cite some of those metrics in support of Merrow. From 1998 to 2002, black 8th grade reading scores increased from an average of 233 to 238. By 2015, they were down to 236. From 1998 to 2002, average 8th grade reading scores for low-income students increased from 229 to 233. In 2015, they remained at 233.

 

Thompson says it is sad that the elites now re-engineering public education are utterly disconnected from the lives and realities of the children who attend those schools or the people who teach in them. They need a reality check, or maybe a course in sociocultural sensitivity training so that they stop stepping on the faces of children and adults whose lives they know nothing about.

 

 

Categories Billionaires, Common Core, District of Columbia, Education Reform, NAEP, Testing

News Flash for the New York Times!

By dianeravitch
November 22, 2015 //
51

News flash! There is a national test that enables us to compare reading and math scores for every state! It is called NAEP. It reports scores by race, ELLs, poverty, gender, disability status, achievement gaps. This is apparently unknown to the Néw York Times and the Secretary of Education, who has said repeatedly that we need Common Core tests to compare states.

The New York Times, America’s newspaper of record, has a story today about Massachusetts’ decision to abandon PARCC, even though its State Commissioner Mitchell Chrster is chairman of the board of PARCC. True or Memorex? Time will tell.

But the story has a serious problem: the opening sentence.

“It has been one of the most stubborn problems in education: With 50 states, 50 standards and 50 tests, how could anyone really know what American students were learning, or how well?”

Later the story has this sentence:

“The state’s rejection of that test sounded the bell on common assessments, signaling that the future will now look much like the past — with more tests, but almost no ability to compare the difference between one state and another.”

What happened to the National Assessment of Educational Progress? It has been comparing all the states and D.C., as well as many cities, since 1992. Has no one at the New York Times ever heard of NAEP?

Categories Common Core, Massachusetts, NAEP, Standardized Testing

A Note to Marc Tucker: Why I Disagree with Your Analysis of NAEP Scores

By dianeravitch
November 11, 2015 //
22

Dear Marc,

I read your latest post about NAEP scores in which you say you are taking a long view. You dismiss the disappointing results of the 2015 NAEP, which showed almost no gains and some declines. You choose instead to look at the Long-Term Trend NAEP, which has been asking identical questions since 1973. You point out that 17-year-old scores are flat since the early 1970s, which persuades you that we are in big trouble.

For readers, let me explain that there are two different versions of NAEP. The one that was recently reported is called “Main NAEP.” Its curriculum framework is updated every seven to 10 years, and the content changes. Main NAEP is offered in every state every other year, in reading and mathematics. It also periodically tests other subjects, such as history, civics, and science. It gives data for individual states for students in fourth and eighth grades, enabling anyone to compare performance from state to state. It also reports on achievement gaps among students who are white and black, white and Hispanic.

Then there is the “Long-Term Trend” NAEP. It is offered every four years. The reading LTT started in 1971, the math in 1973. Unlike Main NAEP, the content almost never changes, although items that are obsolete are deleted (the one deleted item I recall from my time on the governing board of NAEP was about S&H Green Stamps). It breaks out scores by race and gender.

Marc, you note the impressive progress made by students at ages 9 and 13, especially black and Hispanic students. But you then go on to say that at the current rate of improvement, 80% of students would not reach “proficient” for many decades, perhaps more than a century. I have to disagree with you here, because setting NAEP proficient as a goal is as unrealistic as the NCLB mandate that 100% of American students would be proficient by 2014. NAEP proficient is a very high standard; it represents a very high level of achievement. NAEP started measuring state performance in 1992, and 23 years later, Massachusetts is the only state in the nation where as many as 50% of students have reached NAEP proficient. Why set an impossible goal?

It is true that the scores for 17-year-old students have barely moved, but not for the reasons you cite. It is not that students get dumb as they reach senior year, but that they don’t give a hoot about a test that means nothing to them. When I was on the NAEP governing board, we devoted an entire meeting to discussing the problem of motivation for students at age 17 or senior year. Seniors doodled or made patterns on the answer sheets. They didn’t care what their score was because they knew the test didn’t matter. It didn’t affect their grades; it didn’t affect their college prospects. They would never find out how they did. For them, it was a meaningless exercise. The board considered ways to motivate them. Suppose we offered a pizza party to encourage students to care? Suppose we offered cash prizes? We could not agree on a solution to the problem of motivating high school seniors to take seriously a test that didn’t count.

And that is why I am not surprised or alarmed by the test scores of 17-year-old students on a test that they know doesn’t matter to them.

Categories NAEP, Testing

Listen to Carol Burris Talk about NAEP Scores on Public Radio

By dianeravitch
October 28, 2015 //
6

Carol Burris carefully reviewed the NAEP scores. Listen to her interview on public radio. Unlike many commentators, she has the advantage of being an experienced educator and is also executive director of the Network for Public Education.

Categories NAEP, NCLB (No Child Left Behind), Race to the Top, Standardized Testing

Who Set the Passing Marks for the Common Core Tests? A Design for Failure.

By dianeravitch
September 10, 2015 //
26

The passing marks (cut scores) on the Smarter Balanced Assessment were set last November. They were set in such a way that most students were certain to “fail.” The SBAC predicted that most students would fail. The executive director, Joe Wilhoit, quoted in the article below, predicted that “over time, the performance of students will improve.” Maybe they will, maybe they won’t. It is a dirty trick to play on students. The cut scores are close to the “proficient” achievement level in NAEP. In twenty-three years of testing the states, Massachusetts is the only state in the nation in which 50% of students reached the proficient level. That indicates that it will be many years–if ever–until half of the students are able to reach these absurdly high cut scores. If they are used for promotion and graduation in the future, most students will not be promoted and will not graduate.

Catherine Gewertz of Education Week wrote:

In a move likely to cause political and academic stress in many states, a consortium that is designing assessments for the Common Core State Standards released data Monday projecting that more than half of students will fall short of the marks that connote grade-level skills on its tests of English/language arts and mathematics.

 
The Smarter Balanced Assessment Consortium test has four achievement categories. Students must score at Level 3 or higher to be considered proficient in the skills and knowledge for their grades. According to cut scores approved Friday night by the 22-state consortium, 41 percent of 11th graders will show proficiency in English/language arts, and 33 percent will do so in math. In elementary and middle school, 38 percent to 44 percent will meet the proficiency mark in English/language arts, and 32 percent to 39 percent will do so in math.

 
Level 4, the highest level of the 11th grade Smarter Balanced test, is meant to indicate readiness for entry-level, credit-bearing courses in college, and comes with an exemption from remedial coursework at many universities. Eleven percent of students would qualify for those exemptions.
The establishment of cut scores, known in the measurement field as “standard-setting,” marks one of the biggest milestones in the four-year-long project to design tests for the common standards. It is also the most flammable, since a central tenet of the initiative has been to ratchet up academic expectations to ensure that students are ready for college or good jobs. States that adopted the common core have anticipated tougher tests, but the new cut scores convert that abstract concern into something more concrete.

 

Smarter Balanced is one of two main state consortia that are using $360 million in federal funds to develop common-core tests. The other group, the Partnership for Assessment of Readiness for College and Careers, or PARCC, is waiting until next summer—after the tests are administered—to decide on its cut scores. Smarter Balanced officials emphasized that the figures released Monday are estimates, and that states would have “a much clearer picture” of student performance after the operational test is given in the spring.

Categories Common Core, NAEP, Standardized Testing

Hechinger Report: Common Core Tests in New York Widen Achievement Gaps

By dianeravitch
August 20, 2015 //
22

Emmanuel Felton and Sarah Butrymowicz write in the Hechinger Report that students in New York have shown little progress in three years of Common Core teaching and testing. Experts warn that three years may be too short a time line to reach a judgment. Nonetheless, the widening achievement gaps are cause for concern. According to the conventional wisdom, the writers say, scores were supposed to rise as teachers and students became accustomed to the new standards. The reality is different.

Three years into the transition to harder tests, scores across the board have remained low and largely stagnant.
Thirty percent of all fifth-graders passed the English exam, for instance – while just 7 percent of special education students did. In math, 43 percent of all fifth-graders were proficient, but only a quarter of black students were….

The past three years of testing have been rough for New York. Complaints began right away in 2013 when the state switched to the new exam and continued when the scores showed proficiency rates had dropped roughly 24 percentage points in English and 34 percentage points in math. In the subsequent two years, criticism grew – over the stakes attached to the exams, the tests themselves and the standards. A robust “opt-out” movement led by disaffected parents and supported in part by teachers resulted in 20 percent of New York students not taking the exams, up from 5 percent the previous year….

And scores have not improved much in the three years. A Hechinger Report analysis found that English scores were essentially stagnant across the state and math scores went up slightly. White and Asian students, however, drove this increase, while the gulf between black and Latino students and their peers has widened.
In 2013, for example, 30 percent of fifth-graders passed the state math exam. This year, when the vast majority of those students were in seventh grade, 35 percent of seventh-graders passed the test. But while white students went from 36 to 46 percent proficiency, black students only increased from 15 to 17 percent and Hispanic students from 18 to 20 percent…

In addition to looking a lot like last year’s results, these scores also match New York’s results on the National Assessment of Educational Progress (NAEP), often called the Nation’s Report Card, which is considered the gold standard for student exams.

But the alignment with NAEP is precisely the problem. The state exams now consider “NAEP Proficient” to be a “passing mark.” This is utterly absurd. NAEP Proficient was never intended to be a passing mark, nor is it “grade level.” NAEP Proficient represents a high level of achievement. No state has seen as many as 50% of its students reach NAEP Proficient except for Massachusetts. As long as the states continue to use tests whose “cut scores” are aligned with NAEP, a majority of students will be considered “failures.” This is not sustainable. Think of the consequences of failing most students year after year.

Categories Common Core, NAEP, New York, Standardized Testing

NAEP Will Collect Data on Grit and Other Unquantifiable Traits

By dianeravitch
June 30, 2015 //
47

The National Assessment of Educational Progress (NAEP) has been measuring national samples of students in grades 4 and 8 (and sometimes 12) since the early 1970s. It has been measuring state samples since 1992, and began assessing a few urban districts in 2003. It assesses students every two years in reading and math, and every several years in history, science, civics, and other subjects.

 

NAEP has always collected background information, which is self-reported about students’ reading habits, television viewing time, teacher practices, and other quantifiable aspects of tea hing and learning.

 

Now, NAEP will add grit, motivation, and mindset to the background information collected.

 

It will be interesting to see how these noncognitive traits are measured. Will students judge their own grit? Are they good judges of their grit? Will we someday know which states and cities have students with the most grit? And once we know, will officials create courses in how to improve grit?

 

I am reminded of a strange finding that emerged from international background questions two decades ago. Students were asked if they were good in math. Students in nations with the highest test scores said they were not very good in math; students in nations where test scores were middling thought they were really good at math.

 

What does it all mean? I don’t know, but it satisfies someone’s need for more data.

Categories NAEP, Standardized Testing
« Older Entries
Newer Entries »