Archives for category: NAEP
« Older Entries
Newer Entries »

The Error That Caused the New York Test Scores to Collapse

By dianeravitch
August 9, 2013 //
66

Here is the reason for the collapse of test scores in New York City and New York State.

State officials decided that New York test scores should be aligned with the achievement levels of the federally-administered National Assessment of Educational Progress (NAEP).

This is an excerpt from a press release prepared by Mayor Bloomberg’s office:

“The new State test results are in line with previous results for student’s readiness for college and careers and show New York City students have maintained gains made over the past decade. The percentage of New York City students meeting the new, higher bar for proficiency in math (29.6 percent) is similar to the percent of students measured proficient on the 2011 National Assessment for Educational Progress (NAEP) tests (28.0 percent) – up from 20.5 percent on the NAEP test in 2003. The percentage of New York City students meeting the new, higher bar for proficiency in English (26.4 percent) is similar to the results on the most recent NAEP English test (26.5 percent), up from 22.0 percent on the NAEP test in 2003.”

Now, leave aside for the moment the odd fact that the mayor is boasting about the appallingly low percentage of students in New York City who met the new proficiency standards after a decade of his control of the schools. The key point here is that the mayor, his Chancellor Dennis Walcott, Regents’ Chancellor Merryl Tisch, and State Commissioner John King all agreed that the state and city scores should be comparable to the NAEP “proficiency” level.

That is a huge mistake. It explains why the scores are invalid.

The state didn’t just “raise the bar.” It aligned its passing mark to a completely inappropriate model.

The state scores have four levels: level 4 is the highest, level 1 is the lowest. In the present scoring scheme, students who do not reach level 3 and 4 have “failed.”

NAEP has three levels: “Advanced” is the highest (only about 3-8% of students reach this level). “Proficient” is defined by the National Assessment Governing Board as “solid academic performance for each grade assessed. This is a very high level of academic achievement.”). “Basic” is “partial mastery” of the skills and knowledge needed at each grade tested.

“Proficient” on NAEP is what most people would consider to be the equivalent of an A. When I was a member of the NAEP governing board, we certainly considered proficient to be very high level achievement.

New York’s city and state officials have decided that NAEP’s “proficiency” level should be the passing mark.

They don’t understand that a student who is proficient on NAEP has attained “a very high level of academic achievement.”

Any state that expects all or most students to achieve an A on the state tests is setting most students up for failure.

If students need to reach “proficiency” just to pass, there will obviously be a very large number of students who “fail.”

B students and C students will fail.

The NAEP achievement levels have always been controversial. Many researchers and scholarly bodies have said they were unreasonably high and thus “fundamentally flawed.” That term “fundamentally flawed” occurs again and again in the literature of NAEP critics. This article by James Harvey is a good summary of these arguments.

Some on this blog have asked whether NAEP is a criterion-referenced test, and the answer is no. A criterion-referenced test is one that almost everyone can pass if they master the requisite skills. A test to get a drivers’ license is a criterion-referenced test. Anyone who studies the laws can pass the written test and qualify for a drivers’ license.

NAEP is not a criterion-referenced test. Massachusetts is the only state where as much as 50% of the students (and only in fourth grade) are rated proficient in reading. The NAEP tests are not designed to be criterion-referenced tests; they are a mix of questions that are easy, moderate, and difficult.

The achievement levels were created when Checker Finn was chair of NAGB. I think they are defensible if people understand that the achievement levels do not represent grade levels. If the public wants a measure of “grade level,” then “basic” probably comes closest to grade level. “Proficient” is not grade level; as NAGB documents state, it represents “a very high level of academic achievement.”

More important, the NAEP achievement levels were never intended to be measures of grade level, and New York officials are wrong to interpret them as such, especially when they mistakenly use “proficient” as the passing mark.

Any state that uses NAEP “proficient” as its definition of “grade level” is making a huge mistake; it will set the bar unreasonably high and will mislabel many students and misjudge the quality of many schools.

And that is exactly what happened in the New York testing fiasco.

If the state sticks to its present course of using NAEP “proficient” as its passing mark, it will encourage criticism of the Common Core standards as unrealistic and stoke parental outrage about Common Core testing.

People know their children, and they know their own school. The politicians may convince them that American education is floundering (even if it is not), but they can’t convince them that their own child and their own school are “failing” when parents know from their own experience that it is not true.

The corporate reformers now using the Shock Doctrine to bash the schools and disparage students may find that their tactic has backfired. They succeed only in adding fuel to the growing movement to stop the misuse of standardized testing.

What is happening in New York is likely to undermine public confidence in the state’s highest education officials and create new converts to the Opt-Out of Testing movement.

The Shock Doctrine may be a boomerang that helps to bring down the madness of No Child Left Behind, Race to the Top, Common Core, the Pearson empire, and every other part of the reformy enterprise.

New York may have inadvertently created by the most powerful recruiting tool for the Opt Out movement.

Categories Accountability, Common Core, NAEP, New York, Testing

New York Times Loves High-Stakes Testing, No Matter How Absurd the Scoring

By dianeravitch
August 8, 2013 //
6

The New York Times editorial board, which has uncritically endorsed every bad piece of legislation or policy that is based on high-stakes testing, warmly endorses the absurd results of the Common Core tests in New York. It echoes Secretary Duncan in asserting that the tests prove how terrible US public education is.

The Times displays its ignorance of the scoring rubric, in which Commissioner John King decided to align New York’s test scores with those of NAEP.

Any student who is not proficient has failed, according to the inexperienced Mr. King.

King seems not to know that the NAEP definition of proficiency does not demonstrate grade level performance, but a very high level of achievement representing superior performance. In everyday terms, proficient on NAEP is a solid A.

But in John King’s world, anyone who is not proficient has failed.

If New York continues to use this definition of proficiency, in which anything less than an A is failure, the majority of New York students will be failures forever.

This is a recipe for killing public education and destroying children’s lives and crushing teacher morale.

Are you listening, editorial writers at the Times?

Categories Common Core, NAEP, New York, Testing

Warning: NY Aligns Scores to NAEP

By dianeravitch
August 6, 2013 //
82

New York City’s chief academic officer–a testing zealot–here announces that scores will plummet on the new Common Core tests administered last spring for the first time. They will plummet because the state decided to align its standards to NAEP, which are far more demanding than those of any state.

Over the years, many researchers have maintained that the NAEP achievement levels are “fundamentally flawed” and “unreasonably high.” If you google the terms NAEP and “fundamentally flawed,” you will find many articles criticizing the NAEP benchmarks. Here is a good summary.

What you need to know about NAEP achievement levels is that they are not benchmarked to international standards. They are based on the judgment calls of panels made up of people from different walks of life who decide what students in fourth grade and eighth grade should know and be able to do. It is called “the modified Angoff method” and is very controversial among scholars and psychometricians.

Setting the bar so high is one thing when assessing samples at a state and national level, but quite another when it becomes the basis for judging individual students. It is scientism run amok. It is unethical. It sets the bar where only 30-35% can clear it. Why would we do this to the nation’s children?

Nonetheless, these “unreasonably high” standards are now the guidelines for judging the students of Néw York.

Consequently, teachers and parents can expect to be stunned when the scores are released.

The good news is that teachers and schools will not be punished this year. The punishments start next year.

Here is the letter that went to all public schools with grades 3-8 in Néw York City:

From: Suransky Shael
Sent: Monday, August 05, 2013 1:54 PM
Subject: 2013 State Common Core Test Results

Dear Colleagues,

I’m writing to let you know that your school’s performance data on the 2013 State Common Core tests is now available for you to view. It is important to note that this data is embargoed by the State Education Department (SED)—you are not to share this information until Wednesday, after citywide data is released and the embargo is lifted.

As you review this information and prepare to share it with your school community, please keep in mind the context in which students took these new tests.

At its heart, our ongoing transition to the Common Core standards is about equal opportunity. It is about giving all students a fair chance to develop the skills they will need to pursue higher education and a quality job and have options that will lead to successful and happy lives.

As you know best, this shift is not easy, and so we are also making sure it is not punitive. These results will not be used to evaluate teachers this year, and students and schools will not be punished. The new tests are about developing a realistic understanding of where students are on the path to college and career readiness and adjusting support to improve students’ performance. Educators across the City are investing remarkable energy in this work; from this new baseline, we expect performance to increase.

SED has said the results will be similar to the City’s scores on the National Assessment of Educational Progress (NAEP), which also measures being on track for college and career readiness—for the City, similar scores would mean proficiency rates around 25-30 percent. Scores for individual student populations could be lower. These numbers might be familiar—in addition to our NAEP scores, the City’s College Readiness Index is also in this range—but seeing these results may still be jarring at first for you, your school community, and the public.

To access your school’s embargoed results now, you may view the State’s verification reports in L2RPT. After the public release, your school’s results will also be made available through the DOE public website, ATS, ARIS, and ARIS Parent Link; see below for a general timeline of when test results are expected in each system. If you need support accessing your school’s results, contact your network data support liaison.

Data System
Expected Timeline
L2RPT
August 5
DOE public website
August 7 (school-level results only)
ATS
Mid-August, within 1 week of State release
ARIS; ARIS Parent Link
Late August, within 3 weeks of State release
Item Skills Analysis reports (available in ARIS private communities)
September
Note: reports will be available according to tested year and current year enrollment; a version based on early October enrollment will be available in October.

The coming days and weeks will be challenging as we work together to explain these results to students, teachers, families, and the public. We will be providing materials and additional information in Principals’ Weekly to make sure you understand and feel comfortable discussing these results and the work ahead. And we will reiterate, time and time again, that students will not be penalized by these new tests and that they can—with hard work and support from their teachers, principal, and family—reach this new, higher expectation.

Ultimately, no one will be pleased by a measure that is expected to show fewer than 30 percent of students are on track for success after high school. But I deeply believe that this change—and the more accurate understanding that will result—is part of a transition that will benefit thousands of students for years to come, and I thank you for your leadership in supporting your school community through this time.

Best,

Shael

To: Principals of schools with grades 3-8
Cc: All cluster leaders; all network leaders; all superintendents

Categories Common Core, NAEP, New York, New York City, Testing

What is the Goal of Common Core Testing?

By dianeravitch
August 5, 2013 //
81

For months, school officials in many states have warned parents to expect proficiency rates on Common Core-aligned tests to plummet.

They have warned that the proportion of students rated proficient was likely to drop by as much as 30%.

When this happens, it will make public education in America look just as bad as the corporate reformers have been claiming.

When New York administered the first Common Core tests last spring, a copy of one fifth grade test was leaked to a Daily News reporter. She sent it to me and I studied it and concluded that the test questions were similar in difficulty to what was typically seen on an eighth grade NAEP test. I went to the NAEP website, looked at the released items and questions, and ranked the fifth grade test as “difficult” for an eighth grader.

Here is a report that I just received from the testing coordinator of a high-performing school in one of the best districts in New York:

“Just to let you know that because I am my school’s test coordinator I just looked at the scores for the ELA.  We are a “high achieving” school.  Last year only 5 students in grades 3, 4 and 5 got a level 1.  Now it is 32. Approximately 40% of our students scored levels 3 and 4 this year down from about 80% last year.  What does this mean?  Nothing because a test that measures skills that could not possibly be taught and is developmentally too hard is INVALID.”

So why the rush to make the tests so hard that more students will fail?

Rick Hess wrote last fall that many of the “reformers” believe that the terrible results (eagerly anticipated by them) will cause suburban parents to demand “reforms” and an escape from their neighborhood schools.

I can’t help but recall that David Coleman, the architect of the Common Core standards, was the treasurer of the board of Michelle Rhee’s StudentsFirst in its first year. If the Common Core tests produce a collapse of proficiency rates, then it makes Rhee and her attacks on public schools look good. Will everyone run for the exits and demand charters and vouchers?

Sick thought, but inescapable.

 

Categories Coleman, David, Common Core, NAEP, Rhee, Michelle, StudentsFirst, Testing

Are Teachers to Blame for Achievement Gap Between Black and White Students?

By dianeravitch
July 27, 2013 //
56

Joy Resmovits, the education reporter for Huffington Post,  is usually a sharp and thoughtful reporter, but she had a bad day today.

Today she posted an article blaming “bad” teachers for the achievement gap between black and white students.

Along the way, she makes some factual errors. For example, she states that the achievement gap in ninth grade reading narrowed from 1994-2012, from 33 points to 13.

But that is wrong, for two reasons.

First, NAEP doesn’t test ninth grade. It tests fourth and eighth grades.

Second, the achievement gap for eighth grade shrank during that period from 30 points to 25 points.

She says the achievement gap persists because black students get less experienced teachers (Teach for America?) and have less success in raising test scores (tautology, anyone?).

Joy should know that the achievement gap exists before the first day of school in kindergarten.

It is nourished by large socioeconomic differences.

The achievement gap is an opportunity gap.

Black students are far likelier than white students to live in poverty, to miss school because of illness, to live in bad housing, to be homeless, to have less access to medical care, to live with tremendous economic insecurity.

Their families have fewer resources to invest in them.

The fact that there is an achievement gap is not prima facie evidence that those who teach black students are not good teachers.

Frankly, it is not like Joy to sound like Michelle Rhee, Joel Klein, and Arne Duncan.

Joy Resmovits owes an apology to the many thousands of urban teachers who are hard-working, dedicated to their students, and determined to educate them despite the insults hurled their way by politicians and the media.

 

Categories NAEP, Poverty, Teachers and Teaching

The Latest NAEP Report: Don’t Believe What You Read about It

By dianeravitch
June 27, 2013 //
40

Today the federal government released the NAEP 2012 “Trends in Academic Progress.” This is known as the Long-Term Trend report. These tests seldom change in content. They are given every four years to national–not state–samples of students at ages 9-13-17.

The reports say that achievement is stagnant, but it is not true. What is truly stagnant are the scores for the past four years.

There were big achievement gains from 1971-2008 for whites, blacks, and Hispanics, and big achievement gains for students at every age level tested–ages 9, 13, and 17.

From 1971-2008, in reading, black students at age 9 gained 34 points; at age 13, 25 points; at 17, 28 points.

From 1971-2008, white students at age 9 made gains of 14 points; at 13 points, 7 points; at 17, 4 points.

From 1971-2008, Hispanic students at age 9 gained 25 points; at 13, 10 points; at 17, 17 points.

However, for the past four years, from 2008-2012, the scores have been stagnant for every racial and ethnic group and for every age group with the singular exception of Hispanic 13-year-olds and female 13-year-olds.

From 2008-2012, the acme of the high-stakes testing era, there were no gains for black students at ages 9 or 13 or 17.

From 2008-2012, there were no gains for white students at ages 9 or 13 or 17.

From 2008-2012, there no gains for Hispanic students at ages 9 or 17. At 13, Hispanic students gained 7 points.

From 2008-2012, there were no gains for males in any age group.

From 2008-2012, there were no gains for females at ages 9 or 17. At age 13, females gained 3 points.

The lesson of the new report: billions spent on high-stakes testing have had minimal to no effect on test scores.

High-stakes testing has failed.

We need to take a new course.

Categories NAEP

NY Daily News Reveals Top-Secret State Test

By dianeravitch
May 3, 2013 //
60

Yesterday I received an email from a reporter from the New York Daily News asking for my reaction to a bootleg copy of the Pearson-made fifth-grade exam for English Language Arts. This is part of the first tests of the Common Core in the state, administered in recent weeks to students in 3rd through 8th grades. Students spent about 90 minutes per day for three days on the ELA tests and repeated the process the next week in math.

I read the passages and the questions based on them. My reaction was that the difficulty level of the passages and the questions was not age-appropriate. Based on test questions I had reviewed for seven years when I was a member of the NAEP board, it seemed to me that the test was pitched at an eighth grade level. The passages were very long, about twice as long as a typical passage on NAEP for eighth grade. The questions involved interpretation, inference, and required re-reading of the passage for each question.

I suppose that is what the test-makers think of as critical thinking, and it may be, but there are also issues of what is appropriate for fifth-graders, as well as recognition that this is a timed test.

When the article appeared, I was not quoted but others agreed that the exam was above fifth-grade level. Aaron Pallas at Teachers College said the vocabulary was sixth grade. But it was not the vocabulary that was disturbing to me: it was the cognitive load, the expectation that fifth-graders could read and interpret long passages on a timed test. It would be interesting to put this test alongside released items from eighth grade NAEP. I tried doing that yesterday afternoon, and to my eye, most of the questions would be rated as “medium” or “hard” for eighth graders.

Very high-performing students may find the exam easy. I suspect it was beyond the comprehension of average fifth grade students, and extremely hard for students in the bottom half.

If this test is indicative of what is in store, It reinforces my concern that the Common Core will widen the achievement gaps. Struggling students will fail.

And by the way, read the smug, arrogant editorial in the Daily News. The editors think it is just great that many kids will fail. They are sure that the tests will reveal the poor quality of education in the city’s schools. They forget that every student in the city has been educated under mayoral control, for which this editorial board has been a consistent cheerleader. Do they understand the contradiction? Not likely.

Categories Common Core, NAEP, New York, Pearson, Testing

Good News! Excellent Results on NAEP Economics Test

By dianeravitch
April 25, 2013 //
25

The National Assessment of Educational Progress (NAEP) just released the results of its Economics test for high school seniors.

Only 18% of students ranked “below basic,” which surely included high numbers of students who are English language learners and have serious disabilities.

82% are basic or above.

A remarkable 43% of students ranked “proficient” or above.

Proficient is excellent performance. Having served on the NAEP Board for seven years, I believe that a student who is proficient demonstrates A level performance.

3% of students rank “advanced.” This is A++ performance.

In any classroom where 43% of the students earn a solid A, great things are happening.

Congratulations to our high school social studies teachers!

Categories NAEP, Testing, U.S. education

What English Classes Should Look Like in the Common Core Era

By dianeravitch
January 19, 2013 //
31

Carol Jago is an experienced English teacher, author, and former president of the National Council of Teachers of English.

Jago writes here about what high school English classes should look like in the Common a core era.

She served on the NAEP assessment committee that set the ratio of 70-30 for test developers.

Here is the key point:

“What seems to be causing confusion are the comparative recommended percentages for informational and literary text cited in the Common Core’s introduction. These percentages reflect the 2009 NAEP Reading Framework http://www.nagb.org/content/nagb/assets/documents/publications/frameworks/reading09.pdf . I served on that framework committee and can assure you that when we determined that 70% of what students would be asked to read for the 12th grade NAEP reading assessment would be informational, we did not mean that 70% of what students read in senior English should be informational text. The National Assessment for Educational Progress does not measure performance in English class. It measures performance in reading, reading across the disciplines and throughout the school day.”

I would clarify further to say that NAEP was not designed to tell teachers what to teach or how to teach. That ratio of 70-30 is an instruction to test developers, not to teachers.

Categories Common Core, Curriculum, Education Reform, NAEP

A Teacher Has a Question for Hari Sevugan

By dianeravitch
January 10, 2013 //
5

Hari Sevugan, the ex-Obama spokesman and ex-StudentsFirst spokesman, has twice commented on this blog in defense of charters and high-stakes testing. In his comments yesterday, he pointed to Florida as a model of excellence, while putting down Massachusetts as not all that it claims to be. In my response, I compared Florida’s NAEP scores to those of Massachusetts. Massachusetts is consistently #1, while Florida ranks about average among the 50 states. I assume that Hari was promoting Glorida because Michelle Rhee ranked it at the top of her personal report card. It is certainly way ahead of Massachusetts in authorizing charter schools, for-profit charters, vouchers, high-stakes testing, and stripping teachers of tenure.

Today, I received a letter from a teacher in Nashville, who asked me to post the following questions to Hari. If he answers, I will post his reply.

“I am a teacher in Nashville Public Schools, who has been teaching for 14 years. I have to be honest that since I have been working on a Masters in Educational Leadership, current reform policies have been gaining my interest. I read Hari’s response on your message board, and I would like to ask him why he would slam Massachusetts’s NAEP results and in the same response hold TIMSS and PIRLS results for Florida as a progressing miracle.

“The same studies that he and the likes of him quote to put schools down and compare us to higher achieving nations are the same tests he uses to hold up academic progress for states that are using the current GERM model. I am fascinated with his spin and ability to turn the student achievement of a state rejecting (for the most part) GERM and yet in the same breath hold up a state that does not perform near Massachusetts as a model for reform.

“Please, have him explain his answer as to why bashing the progress of Massachusetts yet holding up Florida and Louisiana as the proof reform is working. In this country it is so hard to measure which reform is working due to all of the different reforms taking place. But, I do not believe Hari’s and StudentsFirst type of reform will give us sustainable results. So, this letter is really directed to Hari, I just don’t know how to get it to him.

“I hope all is well with you and the rest of your readers and please continue the good fight. The future of public education is relying on this conversation.”

Categories Florida, Massachusetts, NAEP, Rhee, Michelle
« Older Entries
Newer Entries »