Archives for category: Testing

Strange as it may seem, the best education reporter in New York City works for Rupert Murdoch’s New York Post. Her name is Susan Edelman, and she regularly reports on what is happening in the city’s schools without fear or favor. Unlike the New York Times, where reporters cycle in and out of the education beat, Sue has been writing on the subject for many years.

One of her best articles appeared in 2011, when she revealed the source of the non-existent “New York City Miracle.”

The title: “New York’s School Testing Con.”

Mayor Bloomberg trumpeted the 2009 scores as proof of the success of his mayoralty, as proof that the Legislature was wise to give him total control of the school system, and as reason #1 to re-elect him to a third term (which broke the City Constitution’s two-term limit).

Edelman began:

In a stunningly short time, from 2006 to 2009, New York schools celebrated what was presented as a tremendous turnaround. The number of city students passing statewide math tests in the third through eighth grades surged from 58% to 82%. At the same time, the Big Apple graduation rate rose from 49% to an all-time high of 63% last year.

The figures were miraculous.

They were also, for the most part, a lie.

While the scores have risen, real achievement has lagged. Behind the curtain, an erosion of standards has led to a generation of New Yorkers who have been handed high school diplomas but can’t handle the rigors of college or careers.

A new state report finds just 23% of city grads leave high school ready to succeed in college or the work world. About 75% who enrolled at CUNY community colleges flunked the entrance exam, and must take one or more remedial classes in math, reading and writing.

Many blamed State Commissioner Richard Mills, who set graduation standards so high that he had to lower the bar or face the possibility that most students would not get a diploma.

But others saw a coverup of huge proportions when the 2009 scores went through the roof. In response to the spectacular scores, Regent Betty Rosa asked,

 “Why are we celebrating these scores as a miracle, when there is no miracle?” Rosa said she asked.

Another insider said Big Apple officials were urged not to “exaggerate” the results. But Mayor Bloomberg hailed the increase in 2009 as an “enormous victory.” At the time, he had a lot riding on the scores — he was seeking a third term and pushing for legislation to extend mayoral control of the schools.

City officials “got very angry,” the insider said, when Regents Chancellor Merryl Tisch publicly downplayed the results, citing “troubling gaps” between the stellar state scores and lackluster outcomes on national exams.

Mills has maintained the scoring was backed by his panel of experts. But Rosa and other members of the Board of Regents say he kept them in the dark.

“I basically asked, ‘Who sets the cut scores? How is this determined?’ ” said Rosa, who joined the board in 2008. “There was no real explanation. I never got a straight answer.”

Mills and his testing chief, David Abrams, had rebuffed requests in 2008 to investigate the inflation. Faced with a lack of confidence, Mills was “encouraged” to leave in June 2009, insiders said. He declined to comment last week, saying, “I have nothing to add.”

Many city students soon discovered their Big Apple diploma was little more than a piece of paper.

Jasmine Gary, 18, a graduate of Port Richmond HS on Staten Island, was surprised when she scored a 70 on the Regents math exam.

“I don’t know how I passed, because I failed a lot of math classes,” she said.

She applied to CUNY but bombed on the entrance exam. Now she’s required to take a no-credit, $75 remedial class at Borough of Manhattan Community College, but is catching up. “I learn more here,” she said.

Rossie and Angely Torres, 18-year-old twins from The Bronx, earned 76 and 75 respectively on the math Regents at Philip Randolph HS in Harlem. They, too, take remedial classes at BMCC.

“In high school it was just people talking and the teacher would just give us an assignment. It was just to graduate. But here, people work hard and the teacher is more serious,” Rossie said.

Former Chancellor Joel Klein, who left office several months ago to join News Corp, which owns The Post, declined to be interviewed. But he defended his eight-year record via e-mail sent by a city DOE spokesman.

“We’ve long called for higher standards and . . . we still made real gains,” Klein said.

For instance, city fourth-graders have boosted their scores on national reading tests since 2003, though eighth-grade scores have remained flat.

And NYC has outpaced the state’s other big cities, Buffalo, Rochester, Syracuse and Yonkers, the DOE says. In 2002, New York City’s fourth-grade math results were 27% lower than the statewide average, while the other four cities showed a 31% gap. In 2008, New York City was just 8% behind the rest of the state, while the “big four” were 25% behind.

But the more spectacular results have vanished.

The Board of Regents commissioned a study, led by Harvard professor Daniel Koretz, which concluded in 2009 that the statewide grades three-eight tests had become too easy. Mills’ successor, David Steiner, recruited for his experience in teacher development as dean of Hunter College of Education, was charged with making the 2010 tests more comprehensive and less predictable. He also hoisted the cutoff points, requiring students to do more to pass.

Scores plunged. Just 54% of all city students in grades three-eight showed proficiency in math tests last year, compared with 82% in 2009. Reading proficiency citywide fell from 69% to a dismal 42%.

Even so, the tougher tests continued the practice of giving “partial credit” for wrong answers — or no answer at all — if the kids showed some understanding of the concept or did one step right.

On the fourth-grade test, for instance, a kid who answered that a 2-foot-long skateboard is 48 inches got half-credit for adding 24 and 24 instead of the correct 12 plus 12. “They were giving credit for blatantly wrong things,” said a teacher hired to score the tests.

A state report released this month delivered a new blow. It found that most kids who earn less than 75 on the state Regents English test or 80 on the math exam — 65 is passing for both — must take remedial classes before starting college.

That 65 score is misleading as well. It’s based on an adjustable scale — and the state has whittled down the points needed to pass. Back in 2003, students had to get 61.2% of math questions right for a 65 score, the minimum required for a Regents diploma, and 50.5% of questions right for a 55 score, enough for a “local diploma.” Today, students need just 30 points out of a maximum 87 — or 34.5% — to get a 65 score.

“When Johnny or Jenny comes home with a 65 or 70, their parents might think they’ve mastered about two-thirds of the material. In fact, it’s slightly more than a third,” said Steve Koss, a retired city math teacher who has railed against the bloated test scores. “Sadly, most parents don’t understand how the scoring works. If they knew the truth, many would be outraged at what amounts to a fraud perpetrated against them by state and local education officials.”

Last month, the state launched a shorter English Regents exam, cutting it from two days to one, six hours to three, and four essays to one. Instead of three other essays, kids have to write two “well-developed paragraphs.”

To bring matters to the present, the Regents are now debating whether to retain or discard their storied exams, which students must pass to graduate.

But that’s a topic for another post.

Heather Gautney and Eric Blanc warn in the Guardian the Michael Bloomberg’s ideas about education would be a disaster for the nation. He is the only candidate whose ideas about education are in synch with those of Donald Trump, Betsy DeVos, and Arne Duncan. The authors are both supporting Bernie Sanders.

Affer persuading the legislature to give him total control of the city’s 1.1 million public school students, he hired three non-educators as city Chancellor. One of them, a publisher out of her depth, lasted 95 days.

Like Trump and his inept Secretary of Education Betsy Devos, Bloomberg is a fervent backer of privatizing and dismantling public schools across the country. Education, in their view, should be run like a business.

While other establishment Democrats have begun changing their tune in response to the “Red for Ed” movement, Bloomberg’s campaign spokesman has made it clear that privatization will be a core message of his 2020 presidential run: “Mike has always supported charter schools, he opened a record number of charter schools as mayor of New York City, and he will champion the issue as president.”

Indeed, Bloomberg succeeded in massively expanding privately run but publicly funded charter schools during his term as mayor, increasing their number from 18 to 183. His controversial push to “increase school choice” closed over 100 schools in low-income communities and entrenched New York City’s education system as the most racially segregated in the country…

If anything, the main difference between Bloomberg and Trump is that the former has spent far more of his immense personal fortune to boost corporate “education reform” and local candidates driving this agenda. The New York Times reported last week that Bloomberg has spent millions to promote charters in the state of Louisiana alone. And this is just the tip of the iceberg: Bloomberg’s foundation in 2018 announced its plan to spend $375m to promote charters, merit pay, and the sacking of “failing” teachers, among other reforms.

Bloomberg is also an active promoter of high stakes testing. Despite abundant evidence that an excessive testing regime does little to improve real educational achievement, Bloomberg has vociferously sung the praises of this system in op-eds such as Demand Better Schools, Not Fewer Tests. Accordingly, as mayor he fought for a merit pay system through which teachers’ salaries would be pegged to student test scores.Like Trump and DeVos, Bloomberg has also viciously attacked teacher unions and scapegoated educators. He spent much of his mayoral tenure fighting with the powerful United Federation of Teachers (UFT), which he compared to the National Rifle Association. As he put it, “if the UFT wants it, it ain’t good”.

Actually, Bloomberg has poured money into charter school campaigns across the country, not just in Louisiana. He donated big money to school board races in Los Angeles and a charter referendum in Massachusetts, among many other state and local races.. His daughter Emma is one of three billionaire board members of TFA’s political action arm, called Leadership for Educational Equity.

Though his Republican roots are less evident on some other issues, Bloomberg’s personal and political similarity to Trump will make it very hard for him to win in a general election. Trump’s base remains solid – we need a candidate who can increase turnout by energizing the Democratic base and involving new voters in the political process.

That’s why having Bloomberg as the Democratic party’s standard bearer would make defeating Trump exceedingly difficult. At a moment when a wave of successful teachers’ strikes has captured the imagination of millions and changed the national discussion on education, a Bloomberg nomination would be a sure-fire recipe for demoralizing educators and teachers’ unions, an indispensable bastion of organized labor and the Democratic base.

They conclude:

You can’t win in November without teachers. And nobody should expect educators to be won over to a billionaire who has spent much of his career and fortune demonizing them. If you want to save public schools and defeat Trump, Bloomberg is no choice at all.

From Politico today:

Of all the Democratic candidates, Michael Bloomberg has the worst record on education. His education policies mirrored George W. Bush’s No Child Left Behind. He was fully invested in high-stakes testing, data-based decision-making, closing schools with low scores instead of helping them, opening new schools and then closing those schools, creating selective schools that chose students based on test scores, and opening scores of charter schools. He had sole control of the “Panel on Education Policy,” and warned its members that if they disagreed with him, they would be fired. When some disagreed about his blanket prohibition of “social promotion,” he summarily fired them. He hired three non-educators as chancellor to lead the system (one of them last 95 days). He tried and failed to hire business people and people from other other fields as principals. He stands for testing and privatization of public education. He has funded pro-privatization candidates in local and state school board races around the country.

SCOOP … MIKE BLOOMBERG is airing another national TV ad tying himself to closely BARACK OBAMA. This one is a 30-second spot entitled “Difference,” and it’s chock-full of imagery of BLOOMBERGand OBAMA. The timing of this ad is quite interesting, as it comes in the middle of a massive intraparty squabble between BLOOMBERGand Sen. BERNIE SANDERS (more about that in a second). BLOOMBERG has found plenty of ways to tie himself to OBAMA, the most popular Democrat in America. The 30-second ad

— SCRIPT: “[NARRATOR]: A great president and an effective mayor. Leadership that makes a difference. [OBAMA SPEAKING]: He’s been a leader throughout the country for the past 12 years, Mr. Michael Bloomberg is here. [NARRATOR]: Together they worked to combat gun violence, and again to improve education for every child. [OBAMA]: And I want to thank the mayor of this great city, Mayor Bloomberg, for his extraordinary leadership. And I share your determination to bring this country together to finally make progress for the American people.”

BLOOMBERG also has a new 30-second spot with Judge Judy. …

— LAT WITH THE NUMBERS: “Democratic presidential candidate Michael R. Bloomberg has spent more than $124 million on advertising in the 14 Super Tuesday states, well over 10 times what his top rivals have put into the contests that yield the biggest trove of delegates in a single day. The only other candidate to advertise across most of those states so far is Vermont Sen. Bernie Sanders, who has spent just under $10 million on ads for the March 3 primaries.”

NEWS: BLOOMBERG has qualified for the NBC/MSNBC/Nevada Independent debate Wednesday night in Las Vegas. He’s indicated that he’ll do it, and a brand-new poll suggests that his advertising and publicity blitz has vaulted him into second place nationally.

THE POLL: SANDERS, 31 … BLOOMBERG, 19 … JOE BIDEN, 15 … ELIZABETH WARREN, 12 … AMY KLOBUCHAR, 9 … PETE BUTTIGIEG, 8. NPR/PBS NewsHour/Marist Poll

— BLOOMBERG’S DEBATE PREP, via Chris Cadelago and Sally Goldenberg: “Howard Wolfson, the veteran Democratic strategist who joined Bloomberg’s orbit in 2009 after working on Hillary Clinton’s 2008 presidential race, is playing the role of Bernie Sanders; Julie Wood, Bloomberg’s national press secretary, is depicting Elizabeth Warren; and senior advisers Marc La Vorgna and Marcia Hale are stand-ins for Pete Buttigieg and Amy Klobuchar, respectively. …

“Bloomberg is trying to hone a crisp and energetic appeal to voters that will contrast with Biden — another white, male septuagenarian on stage, according to advisers.”

THE BRAWL right now between BLOOMBERG and SANDERS seems to be the rare internecine fight that benefits everyone involved. It goes something like this: BLOOMBERG whacks BERNIE, delighting the Democratic Party’s large anti-Bernie wing. BERNIE then blasts out a fundraising email to his list of millions. He reminds his supporters that BLOOMBERG is a billionaire who palled around with TRUMP,and the left goes wild, but so do BLOOMBERG supporters, who say only a deep-pocketed billionaire willing to punch can take on the president.

The munificently-funded Thomas B. Fordham Institute, based in D.C., controls Educatuon Policy, graduation requirements, curriculum, and testing in Ohio. Mr. Fordham, for whom the institute is named, had no known interest in education, but his namesake is part of the rightwing ALEC nexus, where contempt for public schools, hatred for unions, contempt for gun control and environmental regulation are reflexive.

Laura Chapman, who lives in Ohio, writes:

 

This numbers game is routinely pushed by the Ohio arm of Thomas B. Fordham Institute/Foundation. Oped’s written by employees at criticize the Fordham routinely criticize teacher unions for pointing out the debilitating affects of poverty on students. In a typical rhetorical move, the Fordham “expert” will find one exceptional school with an “A” rating of the state report card rigged to ensure few schools are rated A. Then when you read in detail, you will see that the most exceptional thing about this school is really rare. The same principal has been there for 18 years, lives in the community, and has an uncommon level of trust from her community, the teachers, and students. Test scores were a byproduct of that not the aim of her work as an educator.

In Ohio, the writer most responsible for this misleading journalism and “research” is Aaron Churchill, the Institute’s Ohio Research Director. The Institute says this: Since 2012, Aaron has worked on “strengthening” Ohio policy on standardized testing and accountability, school evaluation, school funding, educational markets, human-resource policies and charter school sponsorship. He writes for the Fordham’s blog, the Ohio Gadfly Daily and contributes op-eds to the Columbus Dispatch, Cleveland Plain-Dealer, Dayton Daily News, and Cincinnati Enquirer. Aaron previously worked for Junior Achievement.”

He has not an ounce of documented experience in teaching or studies of education as an undergraduate or graduate student. He gets a free pass on almost everything he submits to the Columbus Dispatch, Cleveland Plain-Dealer, Dayton Daily News, and Cincinnati Enquirer. These local newspapers are shrinking and have few if any staff available for questioning this “throughput” of misleading but ready to post news.

John Thompson, historian and retired teacher in Oklahoma, reviews SLAYING GOLIATH. This is the second part of his review.

This is an excerpt of a long and thoughtful review.

This second post will focus on Ravitch’s analysis of the research which predicted the defeat of accountability-driven, charter-driven policies. Perhaps the most striking pattern documented in Slaying Goliath is how they failed in the way that scholars and practitioners anticipated.

Decades of Disruption-driven reform began with the false claim “that American education was failing and the only way to fix it was with standards, tests, competition, and accountability.” As Arne Duncan’s public relations officer and Walton-funded reformer Peter Cunningham said, “We measure what we treasure.”

Ravitch’s response was, “I was taken aback because I could not imagine how to measure what I treasure: my family, my friends, my pets, my colleagues, my work, the art and books I have collected.” And that foreshadows the victory of the Resistance over Goliath. Most educators, patrons, and students agree that children are more than a test score.

No Child Left Behind and the Race to the Top set impossible test score targets. They were based in large part on the weird idea that “no-excuses” behaviorist pedagogies could be quickly “scaled up,” providing poor children of color a ladder to economic equality. Drawing on the tradition of Edward Thorndike and B.F. Skinner, reformers “rigidly prescribed conditioning via punishments and rewards.” Previewing their fatal flaw, Ravitch observes, “Behaviorists, and the Disrupters who mimic them today, lack appreciation for the value of divergent thinking, and the creative potential of variety. And they emphatically discount mere ‘feelings.’”

When educators resisted, corporate reformers became livid and doubled down on the punitive. Perhaps their worst debacle was using value-added teacher evaluations to hold each individual educator accountable for test score growth. It combined inappropriate test outputs with an unreliable and invalid algorithm, the VAM, as a club to enforce compliance. In the short run, it forced educators, who had previously tried to keep their heads down and “monkey wrench” testing mandates to join patrons and students in the Resistance. By 2018, however, pent up anger exploded as teacher strikes spread across the nation.

Today, many or most of Goliath’s coalition have become disenchanted with standardized testing, but their Disruption model can’t function without it. Few have gone as far as Paymon Rouhanifard, the former Camden superintendent who abolished report cards after listening to complaints, and denounced standardized testing as he left the job.

The more common path is to spin their punitive tests as “personalized” learning, and their incentives and disincentives as the “portfolio model.” As Ravitch explains, “A portfolio district is one where the local board (or some entity operating in its stead) acts like a stockbrokerage, holding onto winners (schools with high test scores) and getting rid of losers (schools with low test scores).”

As was also predicted by Campbell’s Law, test-driven accountability (made more intimidating by the dual threat of test-driven competition with charters) led to corruption. The cheating was far greater than just the scandals where adults erased and changed bubble-in answers. Graduation rates were easy to manipulate. For instance, NPR reported a “heartwarming story” in 2017 about a school with 100% graduation rate. A subsequent FBI investigation and a district audit found 1/3rd of the school’s graduates lacked credits and only 42% were on track to graduation.

And that leads to the corruption associated with school choice. Today’s Disrupters seem to be doubling down on charters to drive transformative change. As explained in a previous post, in 1988 Al Shanker saw charters as a path towards innovation. Within two years, however, the promise of win-win experimentation started to be undermined when conservative reformers Terry Moe and John Chubb claimed “choice is a panacea.”

In this case, it was choice-advocate Paul Peterson who predicted the political future. Charters didn’t take off because of the balanced approach of Shanker, but because reformers “radicalized” the concept. And, of course, there was plenty of big bucks available for pushing their radical but false narrative.

Within a decade, a shocking number of non-educators had been convinced by Goliath’s spinsters that the KIPP’s behaviorist model could be scaled up. As Slaying Goliath explains, “The biggest innovation in the charter sector was the invention of ‘no-excuses’ schools.” It took nearly another decade for policy makers to accept the fact that charters get average results except for those with high attrition.” And it took nearly as long to reveal the much greater down sides of charters…

Regardless of whether we’re discussing high-stakes testing, charter expansion, or the other pet theories, we should all heed Ravitch’s most important lesson of the past few decades is that “Reform doesn’t mean reform. It means mass demoralization, chaos, and turmoil. Disruption does not produce better education.”

Slaying Goliath celebrates a great victory for public education and democracy. However, Ravitch reminds us that the Disrupters are still threatening. She compares today’s danger to that which faced a man who decapitated a rattlesnake but who nearly died after being bitten by the detached head.

So, we can’t lower our guard until the principles that inspired the Resistance are safe in our schools.

 

Fred Smith was the testing expert at the New York City Board of Education for many years. After he retired, he became a relentless truth-teller about the flaws of standardized testing and the clever means of distorting the stats to produce the desired results. He currently acts as an unpaid advisor to opt-out parents.

Smith sent this article from 2007 that shows how Mayor Bloomberg and Chancellor Klein played games with the data, in this case blaming “immigrant kids” for a drop in test scores.

Mayor Bloomberg and his schools chancellor, Joel Klein, have reaffirmed that old Mark Twain saying about the three kinds of lies: lies, damn lies and statistics.

Using a PowerPoint presentation filled with glitzy graphs and color charts, Klein reached a new low yesterday by attempting to blame a sharp drop in this year’s third-, fourth- and fifth-grade reading scores on thousands of immigrant pupils.

According to the chancellor, the drop in the lower grade scores was solely because of the federal government’s new requirement that all children classified as English-language learners, or ELLs, must take the regular state tests after being in the country just one year.

Because of that requirement, some 30,000 more ELLs took the state test this year than in 2006, Klein said, and their lower scores dragged down overall city results.

Fred Smith was outraged when he heard Klein’s explanation. Smith, you see, spent three decades analyzing tests for our city’s school system, so he knows a thing or two about how chancellors paint the prettiest picture for the public.

“They never told you that back in 2005, during the mayoral race, the school district quietly increased the number of exemptions for ELL kids and then claimed a record boost in scores,” Smith said.

In 2009, with Bloomberg’s fellow billionaire Meryl Tisch, in charge of the New York Board of Regents, test scores in the city went through the roof. After the mayoral primary election was safely past, the Regents commissioned a report by professors Daniel Koretz and Jennifer Jennings showing that the test questions had become familiar, leading to score inflation, and that the dramatic rise was not real.

Also, in an amusing turn of events, New York City won the Broad award in 2007 as the most improved urban district, right before the NAEP gains were released, showing that the city had made no gains on NAEP.

In 2010, Jennifer Medina of the New York Times wrote about the perils of over reliance on standardized tests and how it affected New York City in particular. 

She wrote:

When New York State made its standardized English and math tests tougher to pass this year, causing proficiency rates to plummet, it said it was relying on a new analysis showing that the tests had become too easy and that score inflation was rampant.

But evidence had been mounting for some time that the state’s tests, which have formed the basis of almost every school reform effort of the past decade, had serious flaws.

The fast rise and even faster fall of New York’s passing rates resulted from the effect of policies, decisions and missed red flags that stretched back more than 10 years and were laid out in correspondence and in interviews with city and state education officials, administrators and testing experts.

The process involved direct warnings from experts that went unheeded by the state, and a city administration that trumpeted gains in student performance despite its own reservations about how reliably the test gauged future student success.

It involved the state’s decision to create short, predictable exams and to release them publicly soon after they were given, making coaching easy and depriving test creators of a key tool: the ability to insert in each test questions for future exams. Next year, for the first time, the tests will not be released publicly.

It involved a national push for numbers-based accountability, begun under President George W. Bush and reinforced by President Obama. And it involved a mayor’s full embrace of testing as he sought to make his mark on the city, and then to get re-elected.

“They just kept upping the stakes with the scores, putting more pressure on the schools but not really looking at what it all means,” said Pedro Noguera, an education professor at New York University who has worked with the city’s Department of Education to help improve struggling schools.

New York has been a national model for how to carry out education reform, so its sudden decline in passing rates may be seen as a cautionary tale. The turnaround has also been a blow to Mayor Michael R. Bloomberg and his chancellor, Joel I. Klein, who despite warnings that a laserlike focus on raising scores could make them less and less reliable, lashed almost every aspect of its school system to them. Schools were graded on how much their scores rose and threatened with being closed if they did not. The scores dictated which students were promoted or left back, and which teachers and principals would receive bonuses.

Even now, the city believes that the way it uses the tests is valid. The mayor and the chancellor have forcefully defended their students’ performance, noting that even after the changes this year, student scores are still better than they were in 2002. They have argued that their students’ progress is more important than the change in the passing rate, and that years of gains cannot be washed away because of a decision in Albany to require more correct answers from every student this year.

The test scores were even used for a new purpose this year: to help determine which teachers should receive tenure.

“This mayor uses data and metrics to determine whether policies are failing or succeeding,” said Howard Wolfson, the deputy mayor for government affairs and communications. He also helped run Mr. Bloomberg’s re-election campaign in 2009, using the city’s historic rise in test scores to make the case for a third term. “We believe that testing is a key factor for determining the success of schools and teachers.”

“Under any standard you look at,” he added, “we have improved the schools.”

But given all the flaws of the test, said Prof. Howard T. Everson of the City University of New York’s Center for Advanced Study in Education, it is hard to tell what those rising scores really meant.

“Teachers began to know what was going to be on the tests,” said Professor Everson, who was a member of a state testing advisory panel and who warned the state in 2008 that it might have a problem with score inflation. “Then you have to wonder, and folks like me wonder, is that real learning or not?”

New Generation of Tests

The problems that plagued New York’s standardized tests can be traced to the origin of the exams.

In 1996, New York set about creating tests for fourth and eighth graders as a way to measure whether schools were doing their jobs. A precursor to the widespread testing brought about by Mr. Bush’s No Child Left Behind law, the tests replaced more basic exams that had been given in the same grades, which simply determined whether students needed remedial instruction. (The city had also given its own tests for many years.)

Teachers pushed back, saying they could gauge their students’ performance better than any mass-produced tests could. “There was a lot of resistance from throughout the education community to having the tests,” said Alan Ray, who was the chief spokesman for the State Education Department in the 1990s and in 2000, and retired this year after overseeing data for the office.

But education officials in New York, and many other states, were coming to the conclusion that some measurement system, no matter how limited, was necessary.

The officials sought advice from dozens of educators across New York to figure out what the tests should encompass, Mr. Ray said. Teachers and principals asked that the standards be specific, to make it clear what they were expected to teach at each grade level, and superintendents pleaded to keep the tests relatively short so that students would not spend days filling in bubbles. The state obliged both requests.

The decision to keep the tests narrow and short — the fifth-grade math test, for example, had 34 questions this year — would have a lasting impact, said Daniel Koretz, a professor at Harvard’s Graduate School of Education who specializes in assessment systems. The same types of questions would be trotted out every year, he said.

“In many cases you could not write an unpredictable question no matter how hard you tried,” Professor Koretz said. He oversaw the study of New York’s tests that led to the state’s conclusion that they had become too easy to pass.

The state also continued making tests public after they were administered. Coupled with the questions’ predictability, the public release of the tests, which started long before the nationwide accountability movement, provided teachers with ready-made practice exams….

A Mayor Chases Results

The state tests’ flaws would not become evident for years. But by 2001, the tests had a champion.

During his first campaign, Mr. Bloomberg said that education was his top priority. He pledged to take control of the city’s public schools, then under the supervision of the Board of Education, which had been ridiculed for budget troubles and stagnant academic performance.

Projecting the image of a bottom-line-oriented, pragmatic businessman, Mr. Bloomberg latched on to test scores as a clear way of seeing just how well students were doing.

“If four years from now reading scores and math scores aren’t significantly better,” Mr. Bloomberg said in a radio interview in 2001, “then I will look in the mirror and say that I have been a failure. I’ve never failed at anything yet, and I don’t plan to fail at that.”

After Mr. Bloomberg persuaded the Legislature to give him control of the schools, he appointed Mr. Klein, a former Justice Department lawyer and media executive, as his chancellor. Mr. Klein was seen as a technocrat who was eager and able to produce tangible results, the kind that could be measured.

Scores in the city and state were on their way up. In 2004, for example, the proportion of fourth graders in the city meeting math standards increased to 68 percent, up 16 percentage points since 2001. Only 42 percent of eighth graders met that mark, but that was still a significant improvement from just a few years earlier. By 2009, that rate would jump nearly 30 points.

“What is encouraging is that for two or three years in a row now, the tests have gone in the same direction — up,” the mayor said on a radio show in October 2004. “So there’s reason to believe we’re headed to the correct place.”

In 2003, Mr. Bloomberg ended the practice of “social promotion” in certain grades, requiring students performing at the lowest levels on the tests be held back unless they attended summer school and showed progress on a retest. That year, Mr. Klein released a list of 200 successful schools, the only places where teachers would not have to follow the citywide math and English curriculums. The list was primarily based on test scores.

More and more of the mayor’s educational initiatives were linked to the scores. They were used to help decide which schools should be closed and replaced with new, smaller schools. The new A-through-F grading system for schools was based primarily on how their students improved on the tests. Teachers and principals earned bonuses of up to $25,000 if their schools’ scores rose. Teachers’ annual evaluations and tenure decisions are partially dependent on test results.

Each new policy was met with denunciations from the teachers’ union or from education experts like Diane Ravitch. Ms. Ravitch, a supporter of standardized testing when she was an adviser to the Clinton and Bush administrations, became one of the biggest critics, arguing that schools were devoting too much time to the pursuit of high scores.

“If they are not learning social studies but their reading scores are going up, they are not getting an education,” Ms. Ravitch said in 2005, as the mayor coasted to re-election.

The mayor and chancellor dismissed these criticisms as the hidebound defenses of an old, failed system devoid of meaningful standards. But some questions were also being raised by people close to the administration.

In the Education Department headquarters on Chambers Street, some officials argued that the A-through-F system of grading schools should incorporate not only the English and math tests, but also the science and social studies exams given by the state. “We wanted to draw this as broadly as possible,” said a former school official who spoke on the condition of anonymity to avoid publicly disagreeing with Mr. Klein.

But after months of running models and tweaking formulas, Mr. Klein decided to stick with the two core subjects. After all, he often argued, if students could not master essential math and English skills, it would be impossible for them to grasp other concepts.

Dr. Noguera, the N.Y.U. education professor and adviser to the city, applauded Mr. Klein for creating a grading system that rewarded improvement from year to year so that schools in poor neighborhoods had the same chance of achieving a good grade as those in wealthier areas.

But it also was risky, Dr. Noguera said. “That got schools fixated on how to raise scores, not looking for more authentic learning,” he said.

Dr. Noguera expressed his views publicly and to some of Mr. Klein’s deputies, but never directly told the chancellor, he said.

Mr. Klein said in recent interviews that while the tests were imperfect, they were still the best measurements available for a school system that previously had no yardsticks. They also were not the only signs proving the city had been making progress, he said: On more difficult federal tests given to a sample of fourth and eighth graders, the city had steadily improved.

And the city’s main goal, he said, was not simply giving out laurels for students’ scoring 3s (“proficient”) and 4s (“advanced”) on the state tests.

Instead, its system of school grades and teacher incentives gave considerable weight to scores that showed improvement from year to year at all levels.

“Nobody else was doing this,” Mr. Klein said. “We never said it was good enough to get to passing and just stay there.”

In 2006, the state added tests for the third, fifth, sixth and seventh grades, in order to align with the requirements of No Child Left Behind. Scores jumped in 2007.

There were improvements at every grade level across the state and in New York City, where 65 percent of all students met state standards in math, an improvement of eight percentage points in one year.

“I’m happy, thrilled — ecstatic, I think, is a better word,” Mr. Bloomberg said at the time. “The hard work going on in our schools is really paying off.”

After Mr. Bloomberg’s first full term as mayor, the new scores seemed to ratify his claims of success. They also raised more alarms.

As a superintendent in the Brownsville section of Brooklyn, Kathleen Cashin had seen several schools improve throughout the early part of the decade. But when she saw the sudden jump, she said, she was shocked.

“I said to my intimate circle of staff, this cannot be possible,” Ms. Cashin recalled. “I knew how much effort and how much planning any little improvement would take, and not all of these schools had done any of it.”

But Ms. Cashin, who retired in February, held her tongue at the time. Asked why she did not take up her concerns with Mr. Klein or his deputies, she said, “I didn’t have their ear.”

A Proposal for a Fix

The following winter, Professor Koretz, of Harvard, and Professor Everson, of CUNY, who was a member of a state testing advisory group, sent a memo to state education officials.

“Research has shown that when educators are pressured to raise scores on conventional achievement tests, some improve instruction, while others turn to inappropriate methods of test preparation that inflate scores,” they wrote in the Feb. 5, 2008, memo. “In some cases, the inflation of scores has been extreme.”

The researchers proposed to devise a kind of audit. While tests tended to be similar from year to year, they would add to each exam some questions that did not resemble those from previous years. If a class performed well on the main section of the test but poorly on the added questions, that would be evidence that scores were inflated by test preparation. If a class performed well on both, the researchers wrote, that teacher might have methods worth emulating.

In addition, they wrote, such a system would give teachers “less incentive to engage in inappropriate test preparation and more incentive to undertake the much harder task of improving instruction.”

State education officials, the professors said, did not give them a hearing.

The 2008 results showed even more large gains — 74 percent of city students were deemed proficient in math, an increase of nine points in one year; and the city’s passing rate in reading was now 58 percent, up from 51 percent two years earlier. Statewide, the passing rates jumped to 81 percent in math and 69 percent in reading.

Professor Koretz and Professor Everson wrote another memo in September 2008, again proposing to create a way to make test results more reliable. But the idea went nowhere….

The city’s Department of Education constantly mines test score data for patterns to show where improvement is happening and where it is needed. In 2008, it noticed an incongruity: Eighth graders who scored at least a 3 on the state math exam had only a 50 percent chance of graduating from high school four years later with a Regents diploma, which requires a student to pass a certain number of tests in various subjects and is considered the minimum qualification for college readiness.

The city realized that the test results were not as reliable as the state was leading people to believe.

Mr. Klein and several of his deputies spoke by phone with Merryl H. Tisch, the vice chancellor of the Board of Regents, and Mr. Mills, trying to persuade them to create a statewide accountability system similar to the city’s, one that gave improvement at least as much weight as the score itself.

The state said it would consider moving to such a system, but would need more time.

Neither the city nor state publicly disclosed the concerns about the scores. By then, students across the state were preparing for the 2009 tests, filling in bubbles on mock answer sheets, using at least three years of previous state tests as guides.

The scores arrived in May, and with them, the bluntest warning yet.

Just before the results were released, a member of the Regents named Betty Rosa called Ms. Tisch, who had recently become chancellor.

Ms. Rosa, who had been a teacher, principal and superintendent in the Bronx for nearly three decades, said the unprecedented high scores simply seemed too good to be true. She suggested the unthinkable: the scores were so unbelievable, she said, that the state should not publicly release them.

“The question was really are we telling the public the truth,” Ms. Rosa said in a recent interview. Ms. Tisch, she said, relayed that she, too, found the scores suspicious, but that it would be impossible to withhold them. “It was like a train that was already in motion and no way to stop it,” Ms. Rosa said.

The English test scores showed 69 percent of city students passing. Mr. Bloomberg called the results “nothing short of amazing and exactly what this country needs.”

“We have improved the test scores in English,” he continued, “and we expect the same results in math in a couple of weeks, every single year for seven years.” Four weeks later, it was announced that 82 percent of city students had passed the math tests.

Because of the widespread improvement in the scores, 84 percent of all public schools received an A in the city’s grading system, something Mr. Klein said he later regretted. This year, the city limited the number of A’s to 25 percent of schools.

The 2009 numbers came out as the mayor was trying to accomplish two goals: to persuade the Legislature to give the mayor control of the schools for another seven years; and to convince city voters that he deserved a third term.

Mr. Bloomberg’s opponent, Comptroller William C. Thompson, had once been president of the Education Board.

“Mike Bloomberg changed that system,” said one of the mayor’s campaign advertisements. “Now, record graduation rates. Test scores up, violence down. So when you compare apples to apples, Thompson offers politics as usual. Mike Bloomberg offers progress.”

In his debates, Mr. Bloomberg hammered home the theme. “If anybody thinks that the schools were better when Bill ran them, they should vote for him,” he said in one face-off. “And if anybody thinks they’re better now, I’d be honored to have their vote.”

Indeed, according to exit polls, 57 percent of those who said education was their primary concern voted for Mr. Bloomberg, who won the election by a five-point margin.

Mr. Wolfson, the deputy mayor and 2009 campaign strategist, said the mayor had no regrets about focusing on the exams as a matter of policy, and during the election.

“What’s the converse?” he said. “The converse is that we don’t test and we have no way of judging success or failure. Either you believe in standards or tests, or you don’t — and life is not like that. There are tests all the time.”

Ms. Tisch, in releasing the 2009 test results, had not heeded Ms. Rosa’s radical request. But the very day she put out the English test results, she began openly acknowledging doubts about the scores, irking the mayor and chancellor, who privately seethed that she was seeking to undermine their success. “As a board, we will ask whether the test is getting harder or easier,” she said.

Although the Regents did not immediately opt to create an entirely new test, Ms. Tisch and David Steiner, the new education commissioner, asked Professor Koretz, who had been rebuffed in previous requests, to analyze the ones that were in use. His conclusion — and that of another researcher, Jennifer L. Jennings — was that the tests had become too easy, and hence the scores were inflated. That led the State Education Department to raise the number of correct answers required to pass each test.

The state intends to rewrite future tests to encompass a broader range of material, and will stop publicly releasing them.

“We came in here saying we have to stop lying to our kids,” Ms. Tisch said in a recent interview. “We have to be able to know what they do and do not know.”

Bloomberg was first elected to the mayoralty in 2001. There was a two-term limit. He ran again in 2005, for what should have been his second and last term, and won easily. In 2009, he used his vast resources to persuade the City Council to vote to give him and themselves a third term. And that he is how he qualified to run for a third term and used his education record as a reason to be re-elected.
Now, after all this investment in testing, test prep, interim assessments, etc. what were the results?
New York City has shown no gains in reading on NAEP from 2003-2019, in either fourth or eighth grades.
Make of it what you will.
If Bloomberg is the Democratic candidate against Trump, I will vote for him.
But please don’t believe the boasting about the New York City education miracle.
It never happened.
An update on some of the individuals mentioned in the New York Times’ 2010 article. Betty Rosa is now Chancellor of the State Board of Regents. Kathleen Cashin is a member of the Board of Regents. Meryl Tisch is now on the board of the State University of New York (which has the power to authorize new charter schools, including those of Eva Moskowitz’s Success Academy chain). David Steiner–now a professor at Johns Hopkins University– served for two years as State Commissioner, during which time he approved Mayor Bloomberg’s choice to succeed Joel Klein as NYC Chancellor, a retired magazine publisher named Cathie Black, who lasted three months. Steiner was also in charge of the State Education Department when it won a Race to the Top grant and committed the state to using student test scores to evaluate teachers, increasing the number of charter schools, and adopting the Common Core standards. These changes, in turn, created the parent-led Opt Out movement, in which parents refused to let their children take the state tests and grew to represent 20% of the eligible students. John King succeeded David Steiner and eventually replaced Arne Duncan in the last year of President Obama’s second term. When Joel Klein stepped down, he hired a Department of Education vendor named Wireless Generation and created a technology company called Amplify. Rupert Murdoch bought Amplify and invested a reputed $1 billion; newspaper stories predicted that Amplify would usher in a new age of hardware and software. However, the biggest sale of Amplify tablets and software was made to Guilford, North Carolina, purchased with Race to the Top funding; it turned into  a disaster when chargers melted and other problems emerged. Guilford canceled the contract. Murdoch, having lost about $500 million, put the company up for sale. Laurene Powell Jobs bought it, and Amplify is now part of her Emerson Collective, selling “personalized learning.” Klein works for an online healthcare company called OSCAR, co-founded by Joshua Kushner, brother of Jared Kushner.

Many researchers were amazed to see that the state of Mississippi had a sharp growth in its fourth-grade reading scores.

Fortunately, the far-right Thomas B. Fordham Institute reveals how this happened.

The surest path to success in fourth-grade reading on NAEP is to hold back third-graders who did not pass the third-grade reading test. It works! It increased Florida’s fourth-grade reading scores. And it worked for Mississippi too! You have to give credit where it is due: Jeb Bush thought up this way to artificially inflate Florida’s NAEP standing. Research has consistently shown that kids who are held back are likelier to drop out of school later, but who cares about them? The scores and ratings are everything! Mississippi holds back a higher percentage of third-graders than any other state. How about those numbers!

One of the bright spots in an otherwise dreary 2019 NAEP report is Mississippi. A long-time cellar dweller in the NAEP rankings, Mississippi students have risen faster than anyone since 2013, particularly for fourth graders. In fourth grade reading results, Mississippi boosted its ranking from forty-ninth in 2013 to twenty-ninth in 2019; in math, they zoomed from fiftieth to twenty-third. Adjusted for demographics, Mississippi now ranks near the top in fourth grade reading and math according to the Urban Institute’s America’s Gradebook report.

So how have they done it? Education commentators have pointed to several possible causes: roll-out of early literacy programs and professional development (Cowen & Forte), faithful implementation of Common Core standards (Petrilli), and focus on the “science of reading” (State Superintendent Carey Wright).

But one key part of Mississippi’s formula has gotten less coverage: holding back low-performing students. In response to the legislature’s 2013 Literacy Based Promotion Act (LBPA), Mississippi schools retain a higher percentage of K–3 students than any other state. (Mississippi-based Bracey Harris of The Hechinger Report is one education writer who has reported on this topic.)

The LBPA created a “third grade gate,” making success on the reading exit exam a requirement for fourth grade promotion. This isn’t a new idea of course. Florida is widely credited with starting the trend in 2003, and now sixteen states plus the District of Columbia have a reading proficiency requirement to pass into fourth grade.

But Mississippi has taken the concept further than others, with a retention rate higher than any other state. In 2018–19, according to state department of education reports, 8 percent of all Mississippi K–3 students were held back (up from 6.6 percent the prior year). This implies that over the four grades, as many as 32 percent of all Mississippi students are held back; a more reasonable estimate is closer to 20 to 25 percent, allowing for some to be held back twice. (Mississippi’s Department of Education does not report how many students are retained more than once.)

Just goes to show: If at first you don’t succeed, fake it.

 

 

Tim Slekar, one of the pioneers of the Resistance invited me to talk with him on his podcast Busted Pencils. 

We talked about SLAYING GOLIATH.

This post was submitted as a comment by a reader who self-identifies as “Democracy”:

There’s little question that the SAT and ACT are marginally “good” at predicting success in college. I’ve made this point here numerous times.

The best predictor of success in college is high school grade point average (including an SAT score doesn’t add much). Moreover, research shows that “the best predictor of both first- and second-year college grades” is unweighted high school grade point average. A high school grade point average “weighted with a full bonus point for AP…is invariably the worst predictor of college performance.”

The College Board, which produces the PSAT, SAT, and Advanced Placement courses and tests, now recommends that schools “implement grade-weighting policies…starting as early as the sixth grade.” Yes, the SIXTH grade! There’s nothing quite like hyping nonsense.

College enrollment specialists say that their research finds the SAT predicts between 3 and 14 percent of freshman-year college grades, and after that nothing. As one commented, “I might as well measure their shoe size.” Matthew Quirk reported this in “The Best Class Money Can Buy:”

“The ACT and the College Board don’t just sell hundreds of thousands of student profiles to schools; they also offer software and consulting services that can be used to set crude wealth and test-score cutoffs, to target or eliminate students before they apply…That students are rejected on the basis of income is one of the most closely held secrets in admissions; enrollment managers say the practice is far more prevalent than most schools let on.”

http://www.theatlantic.com/magazine/archive/2005/11/the-best-class-money-can-buy/4307/2/

The authors of a study in Ohio found the ACT has minimal predictive power. For example, the ACT composite score predicts about 5 percent of the variance in freshman-year Grade Point Average at Akron University, 10 percent at Bowling Green, 13 percent at Cincinnati, 8 percent at Kent State, 12 percent at Miami of Ohio, 9 percent at Ohio University, 15 percent at Ohio State, 13 percent at Toledo, and 17 percent for all others. Hardly anything to get all excited about.

Here is what the authors say about the ACT in their concluding remarks:

“…why, in the competitive college admissions market, admission officers have not already discovered the shortcomings of the ACT composite score and reduced the weight they put on the Reading and Science components. The answer is not clear. Personal conversations suggest that most admission officers are simply unaware of the difference in predictive validity across the tests. They have trusted ACT Inc. to design a valid exam and never took the time (or had the resources) to analyze the predictive power of its various components. An alternative explanation is that schools have a strong incentive – perhaps due to highly publicized external rankings such as those compiled by U.S. News & World Report, which incorporate students’ entrance exam scores – to admit students with a high ACT composite score, even if this score turns out to be unhelpful.”

The study cited on this thread is from a small (a couple of thousand students) study at the University of Alaska. While some its findings confirm what’s already known about the SAT and ACT, some of its findings suggest that another College Board product — ACCUPLACER — might actually be as good as the College Board says it is.

But it isn’t.

The Community College Research Center at Teachers College, Columbia University has done extensive research on ACCUPLACER (tens of thousands of students), and their research finds that ACCUPLACER has “only a weak relationship with educational performance.” Follow-up research found that the vast majority (71 percent) of students who disregarded low placement test scores to take credit classes rather than remedial ones passed the classes.

Author Nicholas Lemann, the Joseph Pulitzer II and Edith Pulitzer Moore Professor of Journalism and Dean Emeritus at Columbia’s School of Journalism, and whose book The Big Test is all about the SAT, said this about it:

“The test has been, you know, fetishized. This whole culture and frenzy and mythology has been built around SATs. Tests, in general, SATs, in particular…”

Princeton Review founder John Katzman was a bit more blunt:

“The SAT is a scam…It has never measured anything. And it continues to measure nothing. And the whole game is that everybody who does well on it, is so delighted by their good fortune that they don’t want to attack it. And they are the people in charge. Because of course, the way you get to be in charge is by having high test scores. So it’s this terrific kind of rolling scam that every so often, somebody sort of looks and says–well, you know, does it measure intelligence? No. Does it predict college grades? No. Does it tell you how much you learned in high school? No. Does it predict life happiness or life success in any measure? No. It’s measuring nothing.”

The amazing thing – as amazing as the fact that some people still believe Trump – is that some people, including lots of teachers and administrators, not to mention students and parents, still buy into the goofiness.

Yong Zhao writes here about the international test called PISA, which is used to rank, rate and stigmatize entire national systems of education. Its scores are based on a standardized test, of course, which contains the usual flaws of such tests.

His article is called “Two Decades of Havoc.”

Scholars have criticized PISA since it started, but once the global horse race started, there was no slowing it down. PISA now drives every nation to compete for higher scores, in a “race to the top” that very few can win. The critics have been ignored.

It is a stupid metric. Should we really long to be like Estonia? Can all of education be boiled down to questions on a test? What do the results tell us about the future? Nothing, really. When the first international test was given in mathematics in 1964, the U.S. came in last. And over the next 50 years, the U.S. economy surpassed the nations with higher math scores.