Archives for category: NAEP

Some guy who works for StudentsFirst–the organization that promotes vouchers and charters and wants to strip teacher of all due process–wrote a criticism of me on Huffington Post because he doesn’t like the way I interpret NAEP data. This is silly because I served on the NAEP board for seven years and know its strengths and limitations. NAEP was designed to serve an audit function, never to be used for high stakes. Like every other standardized test, NAEP reflects socioeconomic status. The kids with the most advantages score at the top, and those with the fewest advantages cluster at the bottom. NAEP is generally known as “the gold standard” because no one knows who will take it, no student takes the whole test, and no one knows how to prepare for it. NAEP scores may reflect demographic changes or other factors.

Here Mercedes Schneider takes him to task for his misinterpretation of what I wrote.

The release of the 2013 NAEP results set off cheering among advocates of corporate reform because DC and Tennessee showed big gains. But, I pointed out, states following exactltly the same formula showed small gains, no gains, or losses.

He missed the point.

Tom Loveless of the Brookings Institution doesn’t like it when politicians play games with education statistics.

In this post, he gives a lesson in the interpretation and misinterpretation of NAEP scores ranking the states.

Editor’s note:  While Diane is on a somewhat reduced blogging schedule, she has invited members of the Education Bloggers Network, a consortium of people who blog about education issues on the national, state or local level to contribute to her blog.  If you are a blogger who supports public education and would like to join the Education Bloggers Network, contact Jonathan Pelto at jonpelto@gmail.com.

This guest blog is written Paul Thomas

During her tenure as Secretary of Education (2005-2009), Margaret Spellings announced that a jump of 7 points in NAEP reading scores from 1999-2005 was proof No Child Left Behind was working. The problem, however, was in the details:

During President George W. Bush’s tenure, NCLB was a corner stone of his agenda, and when then-Secretary Spellings announced that test scores were proving NCLB a success, Gerald Bracey and Stephen Krashen exposed one of two possible problems with the data. Spellings either did not understand basic statistics or was misleading for political gain. Krashen detailed the deception or ineptitude by showing that the gain Spellings noted did occur from 1999 to 2005, a change of seven points. But he also revealed that the scores rose as follows: 1999 = 212; 2000 = 213; 2002 = 219; 2003 = 218 ; 2005 = 219. The jump Spellings used to promote NCLB and Reading First occurred from 2000 to 2002, before the implementation of Reading First. Krashen notes even more problems with claiming success for NCLB and Reading First, including: 

“Bracey (2006) also notes that it is very unlikely that many Reading First children were included in the NAEP assessments in 2004 (and even 2005). NAEP is given to nine year olds, but RF is directed at grade three and lower. Many RF programs did not begin until late in 2003; in fact, Bracey notes that the application package for RF was not available until April, 2002.”

With the 2013 release of NAEP data, then, shouldn’t we be skeptical of Duncan’s rush to claim victory for education reform under Obama?:

This year, Tennessee and the District of Columbia, which have both launched high-profile efforts to strengthen education by improving teacher evaluations and by other measures, showed across-the-board growth on the test compared to 2011, likely stoking more debate. Only the Defense Department schools also saw gains in both grade levels and subjects.

In Hawaii, which has also seen a concentrated effort to improve teaching quality, scores also increased with the exception of fourth grade reading. In Iowa and Washington state, scores increased except in 8th-grade math.

Specifically pointing to Tennessee, Hawaii and D.C., Education Secretary Arne Duncan said on a conference call with reporters that many of the changes seen in these states were “very, very difficult and courageous” and appear to have had an impact.

Duncan’s claims, in fact, have prompted The Wall Street Journal to announce “School Reform Delivers”:

Education Secretary Arne Duncan hailed this year’s National Assessment of Educational Progress (i.e., the nation’s report card) results on Thursday as “encouraging.” That’s true only if you look at Washington, D.C., Tennessee and states that have led on teacher accountability and other reforms….

However, a handful of states did post significant gains, and the District of Columbia and Tennessee stand out. Until very recently, Washington, D.C. was an example of public school failure. Then in 2009 former schools chancellor Michelle Rhee implemented more rigorous teacher evaluations that place a heavy emphasis on student learning. The district also tied pay to performance evaluations and eliminated tenure so that ineffective teachers could be fired.

Between 2010 and 2012, about 4% of D.C. teachers—and nearly all of those rated “ineffective”—were dismissed. About 30% of teachers rated “minimally effective” left on their own, likely because they didn’t receive a pay bump and were warned that they could be removed within a year if they failed to shape up.

Clearing out the deadwood appears to have lifted scores.

As I warned on the release date of NAEP, we should anticipate this careless and unsupported eagerness to use NAEP data as evidence of corporate reform success.

Jim Horn has highlighted that NAEP shows a powerful picture of the growing problem with re-segregation and the entrenched reality of racial and socioeconomic achievement gaps—messages ignored by Duncan. At the very least, then, Duncan is cherry-picking.

Gary Rubinstein has also dismantled the DC “miracle,” and G.F. Brandenburg provides a clear chart showing that DC gains are a continuation of a trend pre-Rhee. As Rubinstein concludes:

I’m still pretty confident that in the long run education reform based primarily on putting pressure on teachers and shutting down schools for failing to live up to the PR of charter schools will not be good for kids or for the country, in general.  I hope politicians won’t accept the first ‘gains’ chart without putting it into context with the rest of the data.

With the USDOE at Duncan’s disposal, it seems careless and inexcusable to make unproven claims that policy has caused test score changes when no one has had time to analyze the data in order to make such claims

Like Spellings, Duncan is showing that he is either unqualified to be Secretary of Education due to a lack of understanding of statistics or that he is willing to place partisan politics above what is best for children and public education. Either way, this is yet another example of failure from the top in the world of education reform and politics.

The indefatigable Bruce Baker is at his best in this post, where he puts the NAEP scores into perspective. As he notes, it is not useful to look at a two-year test score change as a reliable indicator. It is far wiser to look at scores in a longitudinal fashion and, when possible, look at other factors that may affect test scores. Then, too, he notes that the NAEP results do not align well with Michelle Rhee’s scorecard for the states. Some of the states she considered to be tops don’t do well on NAEP, either short term or long term.

Even with Baker’s fine analysis, it makes me uneasy to see this maniacal national and international race to get the highest scores. As long as our policymakers and federal policy continue to ignore the undying factors of child health and well-being, and the well-being of families and communities, the NAEP scores are like shadows on the wall, interesting but a distraction from the more important factors that create the conditions for a good life, including a respect for and love of learning.

While Arne Duncan and ex-Superintendent Tony Bennett were celebrating Indiana’s gains on the 2013 NAEP, researchers at Indiana University said the gains were no different from the state’s performance in past years on NAEP.

“Relative to the 1-point gains in mathematics and reading for the nation as a whole, the 5- and 4-point gains for Indiana fourth-graders appear impressive,” said Peter Kloosterman, the Martha Lea and Bill Armstrong Chair for Teacher Education and a professor of mathematics education. “However, state samples are relatively small, and thus scores tend to fluctuate more than national scores. In 2000, Indiana was 9 points above the national average in math, but that dropped to 4 points above in 2007 and 2009 before going back to 9. In reading, Indiana has fluctuated from 2 to 5 points above the national average since 2000.”

In addition:

“Regarding the latest Grade 8 results, Kloosterman said gains for Indiana students are comparable to recent years.

“Indiana is now 4 points above the national average in mathematics as compared to 2 points in 2011,” he said. “Since 2000, however, Indiana has been as high as 9 points above and as low as 2 points above. In reading, Indiana eighth-graders are now 1 point above the national average, the same as 2011 and within the window of 1 to 4 points above the national average for Indiana since 2000.”

Although Indiana remains above the national average, it is not in the top tier of U.S. students. “In brief, we see substantial gains in mathematics across the nation with fourth- and eighth-graders in 2013 achieving about two grade levels above their counterparts in 1990,” Kloosterman said. “There have been gains in reading at both levels, but they are much less than a grade level. Indiana is consistently above the national average, but not at the level of the highest-performing states. These trends have held throughout all the state and national education policy changes over this period.”

Kloosterman is available to respond to questions about how to interpret the latest NAEP results. He can be reached at 812-855-9715 or klooster@indiana.edu.

A regular reader who calls him- or herself “Democracy” wrote the following in response to my post about the hype and spin surrounding NAEP scores:

“Diane Ravitch writes this: “Anyone who takes them [NAEP scores} seriously is either a sports writer covering education or someone who thinks that education can be reduced to the scores on standardized tests.”

I don’t disagree. But there are, obviously, plenty of educators and citizens, perhaps even most, who do disagree. They buy into goofy arguments made by the testing business (the College Board, the ACT, Pearson, etc.). They spout the “data-driven” nonsense. They think SAT and ACT scores actually measure “learning” and “intelligence.” They believe that Advanced Placement courses really are “better” than other college preparatory classes. They adopt and implement teacher merit pay schemes based on student test scores. They tout the test scores of their graduates, and of their incoming freshman classes.

Who are these people? School superintendents and school board members. Teachers, Guidance counselors. College admissions officers, and college presidents and board of trustees members. Parents, Politicians.

These are the same people who gamely embraced No Child Left Behind, and who had neither cognitive presence, courage, nor professional conviction to oppose it until THEIR schools were directly threatened.

Many of these same people have now latched onto the Common Core, as a new and improved model of school “reform.” Unfortunately, it’s one that seeks to cure a disease (public schooling in “crisis”) that doesn’t exist. In the process, there’s an incredible waste of resources that might have been used to move in a different research-based direction and affect genuine, meaningful educational improvement.

And what about education reporting. It’s woeful. Or worse. People like Tom Friedman toss off dreadfully ignorant stuff on schooling and test scores. Amanda Ripley passes herself off as an “investigative journalist” and educational “expert.” Jay Mathews at The Post continues to push the “AP is better” myth, while his editors continue to heap praise on the Michelle Rhee-Kaya Henderson regime in DC. At The Educated Reporter and The Atlantic, they seem to have been former sports writers.

I appreciate Diane Ravitch’s efforts to help educate and enlighten those who disagree with her take on test scores. There are certainly a lot of them, and it’s quite an undertaking.”

The latest NAEP reports on reading and math have been heralded as evidence for the success of the “reforms” that involve test prep, testing, punishing teachers if scores don’t go up, rewarding them if they do, closing schools, and other versions of the carrot and stick method of school reform.

Here is my one-word comment: Balderdash!

There are just as many states using the same misguided strategies who made few or no gains as there were reformy states making big gains.

If test-and-punish strategies work, why don’t they work everywhere?

D.C., Tennessee, and Indiana raised test scores, but the gains in other reformy states were small or negligible.

Below the national average were hard-driving reformy. States including Colorado, Delaware, Louisiana, Rhode Island, Ohio, Connecticut, and. North Carolina.

That highly reformy state Wisconsin made no gains at all.

Michigan, New Jersey, and Massachusetts actually lost ground.

It is impossible to conclude, as some leaders have, that D.C., Tennessee, and Indiana have the right formula because so many states with exactly the same formula made no progress at all. Some of the states that were unlucky enough to win Race to the Top mandates made little or no gains or lost ground.

As a former member of the NAEP board, let me say that I find this statistical horse race utterly stupid. Are students in D.C. getting a better education than those in Massachusetts? Highly unlikely.

Are the students in the states with the biggest gains getting better education or more test prep?

Let me say it as bluntly as I know how: these state comparisons are stupid and say nothing about the quality of education available in different states. Anyone who takes them seriously is either a sports writer covering education or someone who thinks that education can be reduced to the scores on standardized tests.

Will families rush to enroll their children in the schools of D.C. or Tennessee because of these scores? Don’t be ridiculous.

This reader points out that the leaders of New York State so not understand NAEP achievement levels. They are not grade levels. “Proficiency” on NAEP means superior academic performance. Please, someone, explain the levels to them :

“John King and Merryl Tisch continue to mislead the public or demonstrate a total lack of understanding for NAEP scores.

Today, Chancellor Merryl Tisch and Commissioner John King released a joint statement reiterating their belief that our public schools are faltering (http://www.oms.nysed.gov/press/naep-scores-2013.html). New York State Board of Regents Chancellor Merryl H. Tisch said. “I’m encouraged by the progress I’ve seen in classrooms around the state and the hard work educators are doing to help their students succeed. But the NAEP results for New York students confirm what we already know: our students are not where they should be… The NAEP results are consistent with the findings of several other measures of New York students, including the state’s measurement of college and career readiness (35 percent of students are college and career ready).

The problem is that the Chancellor and Commissioner’s definition of “proficient” is not synonymous with NAEP’s. NAEP defines proficient as solid academic performance and competence over challenging subject-matter knowledge, application of such knowledge to real-world situations, and analytical skills appropriate to the subject matter. They define basic as partial mastery of prerequisite knowledge and skills that are fundamental for proficient work at a given grade. NAEP’s basic is students achieving appropriate grade level performance. (http://nces.ed.gov/nationsreportcard/achievement.aspx)

Therefore according to NAEP scores 70% of our 4th graders and 76% of our 8th graders are performing at grade level in reading. In math, 82% of our 4th graders and 72% of our 8th graders are performing at grade level. These numbers are much more encouraging than the approximately 35% proficiency levels claimed by the new Common Core State Assessments.

Looking at this data I can draw two conclusions. Either our education officials do not understand what the NAEP scores mean or they are determined to misinform the public. Neither scenario is what I expect of the individuals chosen to lead our public schools. It’s time to stand up, ask questions and let our concerns be heard.

-

The NAEP report card for 2013 is out, and “reformers” were quick to declare vindication for their mean-spirited approach of “test and punish.”

Gary Rubenstein examined the report to see what the claims meant and who made gains. D.C. made the most gains but remains the lowest ranking “state” in the nation. Tennessee made big gains and is pushing to meet the national average. Indiana too saw test score gains.

Does this prove that aggressive moves to close schools and crack the whip on teachers is a formula for success?

Well, no.

Gary looks at the scores for students who are poorest and suddenly D.C. falls to the bottom. At the top are states not known for their adherence to the corporate reform strategies. Suddenly Wyoming and Néw England states are at the top.

G.F. Brandenburg weighs in with his take on the D.C. Scores.

What does all this mean? Probably not as much as it appears. Maybe it shows that a national strategy of test, test, test will raise scores. Kids know how to take standardized tests.

But what be more important in relation to our national quality of life is the rate of child poverty. There the news is not good at all. The Southern Education Foundation reported that children living in poverty are now a new majority in public schools in the South and West.

The rise in child poverty will ultimately be more consequential for our nation than NAEP scores.

Governor Chris Christie has made clear that he doesn’t like the public schools in his state. He calls them “failure factories,” as he campaigns for vouchers. (He is a graduate of Livingston High School.) He seems to despise public school teachers. He enjoys berating teachers, especially if they are female. He is one big, tough, strong guy who knows how to put down women.

Melissa Tomlinson is a public school teacher in New Jersey. She went to a Rally for Governor Chris Christie and she held up a sign.

Read this wonderful description on Jersey Jazzman’s blog of Melissa’s courage in confronting a bully.

Her sign said:

“I am a public school teacher.

“We are NOT failing our students.

“N.J. is ranked 3rd in the US.

“Christie’s refusal to finance public education is failing our students.”

She asked him: “Why do you portray our schools as failure factories?” His reply: “Because they are!” He said: “I am tired of you people. What do you want?”

So, the most powerful executive in the state of New Jersey treated this dedicated public school teacher with arrogance, rudeness, and disrespect. She didn’t back down. She had him cornered. She is right. He is wrong. Probably, he knows he is wrong, so he felt compelled to shout her down instead of engaging in civil dialogue.

Melissa Tomlinson was right that Governor Christie has underfunded the schools. He froze the spending that was supposed to be used to repair schools with leaky pipes and mold and crumbling facilities.

But Tomlinson was wrong about one thing: on the 2011 NAEP, New Jersey was second in the nation in reading, behind Massachusetts and tied with Connecticut. In math, New Jersey was second in the nation. Not third, but second.

The districts in New Jersey that are failing are the ones that are controlled by the state, some for decades. The state has no idea what to do other than to hand students and public funds over to private corporations.

As Julia Sass Rubin pointed out in an earlier blog today, the Christie administration has systematically underfunded districts that enroll children of color. It has stripped them of democratic governance. It has overloaded them with charters that skim the best students and increase segregation. Governor Christie praises charter schools that exclude children who have serious disabilities and children who don’t speak English. The state has embarked on a policy of separate and unequal for the districts that are powerless.

Governor Chris Christie should be ashamed of himself for his systematic neglect of the education of New Jersey’s most vulnerable children as well as his rude and disgraceful behavior towards public school teachers. He should stop his war against public education. It will not help him become president. It will be a huge liability.

Here is a blog in California. Governor Christie, your reputation as a bully is going national.

Follow

Get every new post delivered to your Inbox.

Join 95,341 other followers