Archives for category: Klein, Joel

Fred Smith was the testing expert at the New York City Board of Education for many years. After he retired, he became a relentless truth-teller about the flaws of standardized testing and the clever means of distorting the stats to produce the desired results. He currently acts as an unpaid advisor to opt-out parents.

Smith sent this article from 2007 that shows how Mayor Bloomberg and Chancellor Klein played games with the data, in this case blaming “immigrant kids” for a drop in test scores.

Mayor Bloomberg and his schools chancellor, Joel Klein, have reaffirmed that old Mark Twain saying about the three kinds of lies: lies, damn lies and statistics.

Using a PowerPoint presentation filled with glitzy graphs and color charts, Klein reached a new low yesterday by attempting to blame a sharp drop in this year’s third-, fourth- and fifth-grade reading scores on thousands of immigrant pupils.

According to the chancellor, the drop in the lower grade scores was solely because of the federal government’s new requirement that all children classified as English-language learners, or ELLs, must take the regular state tests after being in the country just one year.

Because of that requirement, some 30,000 more ELLs took the state test this year than in 2006, Klein said, and their lower scores dragged down overall city results.

Fred Smith was outraged when he heard Klein’s explanation. Smith, you see, spent three decades analyzing tests for our city’s school system, so he knows a thing or two about how chancellors paint the prettiest picture for the public.

“They never told you that back in 2005, during the mayoral race, the school district quietly increased the number of exemptions for ELL kids and then claimed a record boost in scores,” Smith said.

In 2009, with Bloomberg’s fellow billionaire Meryl Tisch, in charge of the New York Board of Regents, test scores in the city went through the roof. After the mayoral primary election was safely past, the Regents commissioned a report by professors Daniel Koretz and Jennifer Jennings showing that the test questions had become familiar, leading to score inflation, and that the dramatic rise was not real.

Also, in an amusing turn of events, New York City won the Broad award in 2007 as the most improved urban district, right before the NAEP gains were released, showing that the city had made no gains on NAEP.

In 2010, Jennifer Medina of the New York Times wrote about the perils of over reliance on standardized tests and how it affected New York City in particular. 

She wrote:

When New York State made its standardized English and math tests tougher to pass this year, causing proficiency rates to plummet, it said it was relying on a new analysis showing that the tests had become too easy and that score inflation was rampant.

But evidence had been mounting for some time that the state’s tests, which have formed the basis of almost every school reform effort of the past decade, had serious flaws.

The fast rise and even faster fall of New York’s passing rates resulted from the effect of policies, decisions and missed red flags that stretched back more than 10 years and were laid out in correspondence and in interviews with city and state education officials, administrators and testing experts.

The process involved direct warnings from experts that went unheeded by the state, and a city administration that trumpeted gains in student performance despite its own reservations about how reliably the test gauged future student success.

It involved the state’s decision to create short, predictable exams and to release them publicly soon after they were given, making coaching easy and depriving test creators of a key tool: the ability to insert in each test questions for future exams. Next year, for the first time, the tests will not be released publicly.

It involved a national push for numbers-based accountability, begun under President George W. Bush and reinforced by President Obama. And it involved a mayor’s full embrace of testing as he sought to make his mark on the city, and then to get re-elected.

“They just kept upping the stakes with the scores, putting more pressure on the schools but not really looking at what it all means,” said Pedro Noguera, an education professor at New York University who has worked with the city’s Department of Education to help improve struggling schools.

New York has been a national model for how to carry out education reform, so its sudden decline in passing rates may be seen as a cautionary tale. The turnaround has also been a blow to Mayor Michael R. Bloomberg and his chancellor, Joel I. Klein, who despite warnings that a laserlike focus on raising scores could make them less and less reliable, lashed almost every aspect of its school system to them. Schools were graded on how much their scores rose and threatened with being closed if they did not. The scores dictated which students were promoted or left back, and which teachers and principals would receive bonuses.

Even now, the city believes that the way it uses the tests is valid. The mayor and the chancellor have forcefully defended their students’ performance, noting that even after the changes this year, student scores are still better than they were in 2002. They have argued that their students’ progress is more important than the change in the passing rate, and that years of gains cannot be washed away because of a decision in Albany to require more correct answers from every student this year.

The test scores were even used for a new purpose this year: to help determine which teachers should receive tenure.

“This mayor uses data and metrics to determine whether policies are failing or succeeding,” said Howard Wolfson, the deputy mayor for government affairs and communications. He also helped run Mr. Bloomberg’s re-election campaign in 2009, using the city’s historic rise in test scores to make the case for a third term. “We believe that testing is a key factor for determining the success of schools and teachers.”

“Under any standard you look at,” he added, “we have improved the schools.”

But given all the flaws of the test, said Prof. Howard T. Everson of the City University of New York’s Center for Advanced Study in Education, it is hard to tell what those rising scores really meant.

“Teachers began to know what was going to be on the tests,” said Professor Everson, who was a member of a state testing advisory panel and who warned the state in 2008 that it might have a problem with score inflation. “Then you have to wonder, and folks like me wonder, is that real learning or not?”

New Generation of Tests

The problems that plagued New York’s standardized tests can be traced to the origin of the exams.

In 1996, New York set about creating tests for fourth and eighth graders as a way to measure whether schools were doing their jobs. A precursor to the widespread testing brought about by Mr. Bush’s No Child Left Behind law, the tests replaced more basic exams that had been given in the same grades, which simply determined whether students needed remedial instruction. (The city had also given its own tests for many years.)

Teachers pushed back, saying they could gauge their students’ performance better than any mass-produced tests could. “There was a lot of resistance from throughout the education community to having the tests,” said Alan Ray, who was the chief spokesman for the State Education Department in the 1990s and in 2000, and retired this year after overseeing data for the office.

But education officials in New York, and many other states, were coming to the conclusion that some measurement system, no matter how limited, was necessary.

The officials sought advice from dozens of educators across New York to figure out what the tests should encompass, Mr. Ray said. Teachers and principals asked that the standards be specific, to make it clear what they were expected to teach at each grade level, and superintendents pleaded to keep the tests relatively short so that students would not spend days filling in bubbles. The state obliged both requests.

The decision to keep the tests narrow and short — the fifth-grade math test, for example, had 34 questions this year — would have a lasting impact, said Daniel Koretz, a professor at Harvard’s Graduate School of Education who specializes in assessment systems. The same types of questions would be trotted out every year, he said.

“In many cases you could not write an unpredictable question no matter how hard you tried,” Professor Koretz said. He oversaw the study of New York’s tests that led to the state’s conclusion that they had become too easy to pass.

The state also continued making tests public after they were administered. Coupled with the questions’ predictability, the public release of the tests, which started long before the nationwide accountability movement, provided teachers with ready-made practice exams….

A Mayor Chases Results

The state tests’ flaws would not become evident for years. But by 2001, the tests had a champion.

During his first campaign, Mr. Bloomberg said that education was his top priority. He pledged to take control of the city’s public schools, then under the supervision of the Board of Education, which had been ridiculed for budget troubles and stagnant academic performance.

Projecting the image of a bottom-line-oriented, pragmatic businessman, Mr. Bloomberg latched on to test scores as a clear way of seeing just how well students were doing.

“If four years from now reading scores and math scores aren’t significantly better,” Mr. Bloomberg said in a radio interview in 2001, “then I will look in the mirror and say that I have been a failure. I’ve never failed at anything yet, and I don’t plan to fail at that.”

After Mr. Bloomberg persuaded the Legislature to give him control of the schools, he appointed Mr. Klein, a former Justice Department lawyer and media executive, as his chancellor. Mr. Klein was seen as a technocrat who was eager and able to produce tangible results, the kind that could be measured.

Scores in the city and state were on their way up. In 2004, for example, the proportion of fourth graders in the city meeting math standards increased to 68 percent, up 16 percentage points since 2001. Only 42 percent of eighth graders met that mark, but that was still a significant improvement from just a few years earlier. By 2009, that rate would jump nearly 30 points.

“What is encouraging is that for two or three years in a row now, the tests have gone in the same direction — up,” the mayor said on a radio show in October 2004. “So there’s reason to believe we’re headed to the correct place.”

In 2003, Mr. Bloomberg ended the practice of “social promotion” in certain grades, requiring students performing at the lowest levels on the tests be held back unless they attended summer school and showed progress on a retest. That year, Mr. Klein released a list of 200 successful schools, the only places where teachers would not have to follow the citywide math and English curriculums. The list was primarily based on test scores.

More and more of the mayor’s educational initiatives were linked to the scores. They were used to help decide which schools should be closed and replaced with new, smaller schools. The new A-through-F grading system for schools was based primarily on how their students improved on the tests. Teachers and principals earned bonuses of up to $25,000 if their schools’ scores rose. Teachers’ annual evaluations and tenure decisions are partially dependent on test results.

Each new policy was met with denunciations from the teachers’ union or from education experts like Diane Ravitch. Ms. Ravitch, a supporter of standardized testing when she was an adviser to the Clinton and Bush administrations, became one of the biggest critics, arguing that schools were devoting too much time to the pursuit of high scores.

“If they are not learning social studies but their reading scores are going up, they are not getting an education,” Ms. Ravitch said in 2005, as the mayor coasted to re-election.

The mayor and chancellor dismissed these criticisms as the hidebound defenses of an old, failed system devoid of meaningful standards. But some questions were also being raised by people close to the administration.

In the Education Department headquarters on Chambers Street, some officials argued that the A-through-F system of grading schools should incorporate not only the English and math tests, but also the science and social studies exams given by the state. “We wanted to draw this as broadly as possible,” said a former school official who spoke on the condition of anonymity to avoid publicly disagreeing with Mr. Klein.

But after months of running models and tweaking formulas, Mr. Klein decided to stick with the two core subjects. After all, he often argued, if students could not master essential math and English skills, it would be impossible for them to grasp other concepts.

Dr. Noguera, the N.Y.U. education professor and adviser to the city, applauded Mr. Klein for creating a grading system that rewarded improvement from year to year so that schools in poor neighborhoods had the same chance of achieving a good grade as those in wealthier areas.

But it also was risky, Dr. Noguera said. “That got schools fixated on how to raise scores, not looking for more authentic learning,” he said.

Dr. Noguera expressed his views publicly and to some of Mr. Klein’s deputies, but never directly told the chancellor, he said.

Mr. Klein said in recent interviews that while the tests were imperfect, they were still the best measurements available for a school system that previously had no yardsticks. They also were not the only signs proving the city had been making progress, he said: On more difficult federal tests given to a sample of fourth and eighth graders, the city had steadily improved.

And the city’s main goal, he said, was not simply giving out laurels for students’ scoring 3s (“proficient”) and 4s (“advanced”) on the state tests.

Instead, its system of school grades and teacher incentives gave considerable weight to scores that showed improvement from year to year at all levels.

“Nobody else was doing this,” Mr. Klein said. “We never said it was good enough to get to passing and just stay there.”

In 2006, the state added tests for the third, fifth, sixth and seventh grades, in order to align with the requirements of No Child Left Behind. Scores jumped in 2007.

There were improvements at every grade level across the state and in New York City, where 65 percent of all students met state standards in math, an improvement of eight percentage points in one year.

“I’m happy, thrilled — ecstatic, I think, is a better word,” Mr. Bloomberg said at the time. “The hard work going on in our schools is really paying off.”

After Mr. Bloomberg’s first full term as mayor, the new scores seemed to ratify his claims of success. They also raised more alarms.

As a superintendent in the Brownsville section of Brooklyn, Kathleen Cashin had seen several schools improve throughout the early part of the decade. But when she saw the sudden jump, she said, she was shocked.

“I said to my intimate circle of staff, this cannot be possible,” Ms. Cashin recalled. “I knew how much effort and how much planning any little improvement would take, and not all of these schools had done any of it.”

But Ms. Cashin, who retired in February, held her tongue at the time. Asked why she did not take up her concerns with Mr. Klein or his deputies, she said, “I didn’t have their ear.”

A Proposal for a Fix

The following winter, Professor Koretz, of Harvard, and Professor Everson, of CUNY, who was a member of a state testing advisory group, sent a memo to state education officials.

“Research has shown that when educators are pressured to raise scores on conventional achievement tests, some improve instruction, while others turn to inappropriate methods of test preparation that inflate scores,” they wrote in the Feb. 5, 2008, memo. “In some cases, the inflation of scores has been extreme.”

The researchers proposed to devise a kind of audit. While tests tended to be similar from year to year, they would add to each exam some questions that did not resemble those from previous years. If a class performed well on the main section of the test but poorly on the added questions, that would be evidence that scores were inflated by test preparation. If a class performed well on both, the researchers wrote, that teacher might have methods worth emulating.

In addition, they wrote, such a system would give teachers “less incentive to engage in inappropriate test preparation and more incentive to undertake the much harder task of improving instruction.”

State education officials, the professors said, did not give them a hearing.

The 2008 results showed even more large gains — 74 percent of city students were deemed proficient in math, an increase of nine points in one year; and the city’s passing rate in reading was now 58 percent, up from 51 percent two years earlier. Statewide, the passing rates jumped to 81 percent in math and 69 percent in reading.

Professor Koretz and Professor Everson wrote another memo in September 2008, again proposing to create a way to make test results more reliable. But the idea went nowhere….

The city’s Department of Education constantly mines test score data for patterns to show where improvement is happening and where it is needed. In 2008, it noticed an incongruity: Eighth graders who scored at least a 3 on the state math exam had only a 50 percent chance of graduating from high school four years later with a Regents diploma, which requires a student to pass a certain number of tests in various subjects and is considered the minimum qualification for college readiness.

The city realized that the test results were not as reliable as the state was leading people to believe.

Mr. Klein and several of his deputies spoke by phone with Merryl H. Tisch, the vice chancellor of the Board of Regents, and Mr. Mills, trying to persuade them to create a statewide accountability system similar to the city’s, one that gave improvement at least as much weight as the score itself.

The state said it would consider moving to such a system, but would need more time.

Neither the city nor state publicly disclosed the concerns about the scores. By then, students across the state were preparing for the 2009 tests, filling in bubbles on mock answer sheets, using at least three years of previous state tests as guides.

The scores arrived in May, and with them, the bluntest warning yet.

Just before the results were released, a member of the Regents named Betty Rosa called Ms. Tisch, who had recently become chancellor.

Ms. Rosa, who had been a teacher, principal and superintendent in the Bronx for nearly three decades, said the unprecedented high scores simply seemed too good to be true. She suggested the unthinkable: the scores were so unbelievable, she said, that the state should not publicly release them.

“The question was really are we telling the public the truth,” Ms. Rosa said in a recent interview. Ms. Tisch, she said, relayed that she, too, found the scores suspicious, but that it would be impossible to withhold them. “It was like a train that was already in motion and no way to stop it,” Ms. Rosa said.

The English test scores showed 69 percent of city students passing. Mr. Bloomberg called the results “nothing short of amazing and exactly what this country needs.”

“We have improved the test scores in English,” he continued, “and we expect the same results in math in a couple of weeks, every single year for seven years.” Four weeks later, it was announced that 82 percent of city students had passed the math tests.

Because of the widespread improvement in the scores, 84 percent of all public schools received an A in the city’s grading system, something Mr. Klein said he later regretted. This year, the city limited the number of A’s to 25 percent of schools.

The 2009 numbers came out as the mayor was trying to accomplish two goals: to persuade the Legislature to give the mayor control of the schools for another seven years; and to convince city voters that he deserved a third term.

Mr. Bloomberg’s opponent, Comptroller William C. Thompson, had once been president of the Education Board.

“Mike Bloomberg changed that system,” said one of the mayor’s campaign advertisements. “Now, record graduation rates. Test scores up, violence down. So when you compare apples to apples, Thompson offers politics as usual. Mike Bloomberg offers progress.”

In his debates, Mr. Bloomberg hammered home the theme. “If anybody thinks that the schools were better when Bill ran them, they should vote for him,” he said in one face-off. “And if anybody thinks they’re better now, I’d be honored to have their vote.”

Indeed, according to exit polls, 57 percent of those who said education was their primary concern voted for Mr. Bloomberg, who won the election by a five-point margin.

Mr. Wolfson, the deputy mayor and 2009 campaign strategist, said the mayor had no regrets about focusing on the exams as a matter of policy, and during the election.

“What’s the converse?” he said. “The converse is that we don’t test and we have no way of judging success or failure. Either you believe in standards or tests, or you don’t — and life is not like that. There are tests all the time.”

Ms. Tisch, in releasing the 2009 test results, had not heeded Ms. Rosa’s radical request. But the very day she put out the English test results, she began openly acknowledging doubts about the scores, irking the mayor and chancellor, who privately seethed that she was seeking to undermine their success. “As a board, we will ask whether the test is getting harder or easier,” she said.

Although the Regents did not immediately opt to create an entirely new test, Ms. Tisch and David Steiner, the new education commissioner, asked Professor Koretz, who had been rebuffed in previous requests, to analyze the ones that were in use. His conclusion — and that of another researcher, Jennifer L. Jennings — was that the tests had become too easy, and hence the scores were inflated. That led the State Education Department to raise the number of correct answers required to pass each test.

The state intends to rewrite future tests to encompass a broader range of material, and will stop publicly releasing them.

“We came in here saying we have to stop lying to our kids,” Ms. Tisch said in a recent interview. “We have to be able to know what they do and do not know.”

Bloomberg was first elected to the mayoralty in 2001. There was a two-term limit. He ran again in 2005, for what should have been his second and last term, and won easily. In 2009, he used his vast resources to persuade the City Council to vote to give him and themselves a third term. And that he is how he qualified to run for a third term and used his education record as a reason to be re-elected.
Now, after all this investment in testing, test prep, interim assessments, etc. what were the results?
New York City has shown no gains in reading on NAEP from 2003-2019, in either fourth or eighth grades.
Make of it what you will.
If Bloomberg is the Democratic candidate against Trump, I will vote for him.
But please don’t believe the boasting about the New York City education miracle.
It never happened.
An update on some of the individuals mentioned in the New York Times’ 2010 article. Betty Rosa is now Chancellor of the State Board of Regents. Kathleen Cashin is a member of the Board of Regents. Meryl Tisch is now on the board of the State University of New York (which has the power to authorize new charter schools, including those of Eva Moskowitz’s Success Academy chain). David Steiner–now a professor at Johns Hopkins University– served for two years as State Commissioner, during which time he approved Mayor Bloomberg’s choice to succeed Joel Klein as NYC Chancellor, a retired magazine publisher named Cathie Black, who lasted three months. Steiner was also in charge of the State Education Department when it won a Race to the Top grant and committed the state to using student test scores to evaluate teachers, increasing the number of charter schools, and adopting the Common Core standards. These changes, in turn, created the parent-led Opt Out movement, in which parents refused to let their children take the state tests and grew to represent 20% of the eligible students. John King succeeded David Steiner and eventually replaced Arne Duncan in the last year of President Obama’s second term. When Joel Klein stepped down, he hired a Department of Education vendor named Wireless Generation and created a technology company called Amplify. Rupert Murdoch bought Amplify and invested a reputed $1 billion; newspaper stories predicted that Amplify would usher in a new age of hardware and software. However, the biggest sale of Amplify tablets and software was made to Guilford, North Carolina, purchased with Race to the Top funding; it turned into  a disaster when chargers melted and other problems emerged. Guilford canceled the contract. Murdoch, having lost about $500 million, put the company up for sale. Laurene Powell Jobs bought it, and Amplify is now part of her Emerson Collective, selling “personalized learning.” Klein works for an online healthcare company called OSCAR, co-founded by Joshua Kushner, brother of Jared Kushner.

Jan Resseger writes here about the damage that “portfolio districts” do to students, schools, and communities. The original concept for “portfolio districts” was developed by Paul Hill of the Gates-funded Center for Reinventing Public Educatuon at the University of Washington. The fundamental idea was that the school board would act like a stock portfolio manager, closing low-performing schools, replacing them with charter schools, keeping open the schools with high test scores. Students would choose where to go to school. The concept was adopted by many districts as the latest thing, and many beloved neighborhood schools serving black and brown communities were shuttered. If their replacement got low scores, it was also closed. The students were collateral damage.

She writes:

Mayor Michael Bloomberg and Joel Klein launched this scheme in New York City by creating district-wide school choice, breaking up large comprehensive high schools into small schools with curricular specialties, encouraging the opening of a large number of charter schools, co-locating many schools—small specialty public schools along with charter schools—into the same buildings.  Those running the school district would consider all of these schools of choice as if they were investments in a stock portfolio. The district would hold on to the successful investments and phase out those whose test scores were low or which families didn’t choose.

Portfolio school reform has created collateral damage across the school districts which have experimented with the idea. After the Chicago Public Schools, another district managed by portfolio school reform theory, closed 50 schools at the end of the 2013 school year, the University of Chicago’s Consortium on School Research, and separately a University of Chicago sociologist, Eve Ewing tracked widespread community grieving when neighborhoods lost the public school institutions that had anchored their neighborhoods.

But there have been other kinds of collateral damage beyond the tragedy of school closures. In a new piece for the NY Times, Eliza Shapiro documents how district-wide school choice in New York City has contributed to inequity along with racial and segregation.

One problem is inequitable access to information. Parents who can afford to pay for consultants and who have the skills and position to understand how to navigate the system are able to privilege their own children with access to the schools widely thought to be desirable.  Shapiro explains: “There is a trick to getting to the front of the lines that clog sidewalks outside New York City’s top public high schools each fall. Parents who pay $200 for a newsletter compiled by a local admissions consultant know that they should arrive hours ahead of the scheduled start time for school tours. On a recent Tuesday, there were about a hundred mostly white parents queued up at 2:30 p.m. in the spitting rain outside of Beacon High School, some toting snacks and even a few folding chairs for the long wait. The doors of the highly selective, extremely popular school would not open for another two hours for the tour. Parents and students who arrived at the actual start time were in for a surprise. The line of several thousand people had wrapped around itself, stretching for three midtown Manhattan blocks.”

Resseger adds:

My own children graduated from a racially and economically diverse public high school in Cleveland Heights, Ohio.  Articles like Shapiro’s cause me to appreciate our family’s privilege in a way I had never really previously considered.  From the time they entered Kindergarten, our children knew they would someday go to the big high school at the corner of Cedar and Lee.  At a week-long summer music camp in our school district, middle school students play side-by-side with some of the members of the high school band and orchestra. Our daughter learned to know the high school tennis coach when he worked with younger students in the city recreation program. And the summer before his high school freshman year, our son, knowing that the high school cross country team worked out in a city park during August, went to the park and asked the coach if he could start working out with the team. High school for our children was a natural, predictable, and exciting transition. How lucky we were.

 

When Jan Resseger writes, she does so with authority and clarity.

In this essay, she explains why she will not vote for Michael Bloomberg, based on his record of disrespecting educators in New York City when he was mayor. Bloomberg as mayor employed all the same principles as No Child Left Behind: testing, accountability, school closings, charter schools, school choice, all based on “the business model.”

She writes:

Michael Bloomberg does have a long education record. Bloomberg served as New York City’s mayor from January of 2002 until December of 2013. In 2002, to accommodate his education agenda, Bloomberg got the state legislature to create mayoral governance of NYC’s public schools. In this role, Michael Bloomberg and his appointed schools chancellor, Joel Klein were among the fathers of what has become a national wave of corporate, accountability-based school reform. Bloomberg is a businessman, and Joel Klein was a very successful attorney. Neither had any experience as an educator. They took aggressive steps to run the NYC school district, with 1.1 million students, like a business. Their innovations included district-wide school choice, rapid expansion of charter schools, co-location of a bunch of small charter and traditional schools into what used to be comprehensive high schools, the phase out and closure of low-scoring schools, evaluation of schools by high stakes standardized test scores, the assignment of letter grades to schools based on their test scores, and a sort of merit pay bonus plan for teachers.

In her 2018 book, After the Education Wars, Andrea Gabor, the New York business journalist and journalism professor, comments on Bloomberg’s educational experiment: “The Bloomberg administration embraced the full panoply of education-reform remedies. It worshiped at the altar of standardized tests and all manner of quantitative analysis. The Bloomberg administration also had a penchant for reorganizations that seemed to create more disruption than continuous improvement among its 1.1 million students and 1,800 schools.” ( After the Education Wars, p. 75)

Gabor describes Bloomberg’s expansion of charter schools: “Harlem, in particular, has become the center of an unintentional educational experiment—one that has been replicated in neighborhoods and cities around the country.  During the Bloomberg years, when close to a quarter of students in the area were enrolled in charter schools, segregation increased, as did sizable across-the-board demographic disparities among the students who attended each type of school. An analysis of Bloomberg-era education department data revealed that public open-enrollment elementary and middle schools have double—and several have triple—the proportion of special needs kids of nearby charter schools. The children in New York’s traditional public schools are much poorer than their counterparts in charter schools. And public schools have far higher numbers of English language learners… In backing charter schools Bloomberg and other advocates pointed to one clear benefit: charters, it was widely accepted, would increase standardized test scores. However, years of studies showed little difference between the test-score performance of students in charter schools and those in public schools.” After the Education Wars, p. 95)

And there is more. Open the link and read it to understand why the “business model” did not work.

Billionaire Michael Bloomberg spoke to the national convention of the NAACP about why they should believe in the saving power of privately managed charter schools. He tried to persuade them to rescind their brave 2016 resolution calling for a moratorium on new charters.

This thoughtful report explains why the NAACP called for a moratorium. 

The NAACP deserves our thanks for its resolution and should not back down from its principles, which represent the views of its members, based on hearings in seven cities and long, careful deliberations.

The major conclusions of its resolution:

We are calling for a moratorium on the expansion of the charter schools at least until such time as:

(1) Charter schools are subject to the same transparency and accountability standards as public schools
(2) Public funds are not diverted to charter schools at the expense of the public school system
(3) Charter schools cease expelling students that public schools have a duty to educate and
(4) Charter schools cease to perpetuate de facto segregation of the highest performing children from those whose aspirations may be high but whose talents are not yet as obvious.

Historically the NAACP has been in strong support of public education and has denounced movements toward privatization that divert public funds to support non-public school choices.

“We are moving forward to require that charter schools receive the same level of oversight, civil rights protections and provide the same level of transparency, and we require the same of traditional public schools,” Chairman Brock said. “Our decision today is driven by a long held principle and policy of the NAACP that high quality, free, public education should be afforded to all children.”

Unlike the NAACP, Bloomberg believes in charter schools, along with other billionaires, including the Waltons, the Koch brothers, and the DeVos family. He has funded rightwing candidates across the nation to promote charters; he has also funded candidates who favor vouchers, such as a hard-right school board in Douglas County, Colorado, and in Louisiana, where one of his protégés, State Superintendent John White, is a strong voucher supporter.

Speaking recently to the NAACP, Bloomberg boasted about dramatic gains for black and Hispanic students during his 12 years in office. While he was in office, he boasted that he had cut the achievement gap between black and whites students in half. At his recent speech to the NAACP, he said he reduced it by 20 percent. Neither claim is true. On the National Assessment of Educational Progress, the gap between blacks and whites on eighth grade mathematics was 36 points in 2003 (when he began his education policies) and 38 points in 2013 (the end of his mayoralty). On the NAEP test of eighth grade reading, the gap was 25 points in 2003, 22 points in 2013, but jumped to 29 points in 2015. If he succeeded in reducing the gap, it should have been on a steady downward trajectory. It was not, and it was certainly not cut by 50 percent or 20 percent.

Bloomberg did not mention to the NAACP the many selective high schools he opened whose admission requirements narrowed opportunities for black and brown students (an article in Chalkbeat in 2016 referred to “staggering academic segregation” in the city’s high schools, noting that “over half the students who took and passed the eighth-grade state math exam in 2015 wound up clustered in less than 8 percent of city high schools. The same was true for those who passed the English exam.”

Nor did he did mention the ongoing decline in the number of black and Hispanic students who qualified for the city’s most selective high schools on his watch. The city’s most selective high school, Stuyvesant, has 3,300 students; only 29 are black. Of the 895 offered admission to Stuyvesant this fall, only 7 are black. The decline did not start with Bloomberg, but his policies accelerated the trend of declining enrollment of black and Hispanic students in the elite high schools. He even added more elite high schools. Worse, he raised the entry standards for the gifted programs in the elementary schools that prepare students to apply for the selective high schools, a move that was devastating to black and Hispanic students.

In 2007, Bloomberg’s Department of Education decided to raise the score needed to get into a gifted program, a decision that dramatically reduced the number of black and Hispanic students qualified to enter these programs. Chancellor Joel Klein announced that the city intended to standardize admissions to gifted and talented programs across the city. In the future, Klein said, only those who scored in the top 5% on a standardized test would be admitted. Up until that time, local districts made their own decisions about admissions to gifted programs. Local districts objected to Klein’s new policy, and educators and parents warned that the high cut score would disadvantage black and Hispanic children.

Klein and Bloomberg didn’t listen.

They were wrong.

By 2008, before the program launched, Klein eased the 95% cutoff, lowering it to 90%. Nonetheless, the proportion of minority students who enrolled in gifted and talented programs plummeted.

When New York City set a uniform threshold for admission to public school gifted programs last fall, it was a crucial step in a prolonged effort to equalize access to programs that critics complained were dominated by white middle-class children whose parents knew how to navigate the system.

The move was controversial, with experts warning that standardized tests given to young children were heavily influenced by their upbringing and preschool education, and therefore biased toward the affluent.

Now, an analysis by The New York Times shows that under the new policy, children from the city’s poorest districts were offered a smaller percentage than last year of the entry-grade gifted slots in elementary schools. Children in the city’s wealthiest districts captured a greater share of the slots.

The disparity is so stark that some gifted programs opened by Mayor Michael R. Bloomberg in an effort to increase opportunities in poor and predominantly minority districts will not fill new classes next year. In three districts, there were too few qualifiers to fill a single class.

The new policy relied on a blunt cutoff score on two standardized tests. According to the analysis, 39.2 percent of the students who made the cutoff live in the four wealthiest districts, covering the Upper East Side, the Upper West Side, Staten Island and northeast Queens. That is up from 24.9 percent last year, even though those districts make up 14.2 percent of citywide enrollment in the entry-level grades: kindergarten or first grade, depending on the district.

The total enrollment in gifted classes was not only whiter and more Asian, but the total enrollment was cut in half.

The number of children entering New York City public school gifted programs dropped by half this year from last under a new policy intended to equalize access, with 28 schools lacking enough students to open planned gifted classes, and 13 others proceeding with fewer than a dozen children.

The policy, which based admission on a citywide cutoff score on two standardized tests, also failed to diversify the historically coveted classes, according to a New York Times analysis of new Education Department data.

In a school system in which 17 percent of kindergartners and first graders are white, 48 percent of this year’s new gifted students are white, compared with 33 percent of elementary students admitted to the programs under previous entrance policies. The percentage of Asians is also higher, while those of blacks and Hispanics are lower.

Faced with the fact that the standardized test with a high cut score was excluding black and brown children and shuttering G&T programs in poor communities, the Bloomberg administration did not change the policy.

The policies that Bloomberg put in place continue to determine entrance to gifted and talented programs. For savvy white parents, a place in a G&T program is highly coveted because it promises small classes, smart peers, and special treatment. Getting into one of those programs is very difficult, even for savvy white and Asian parents. Many parents invest in tutoring and test prep to get their four-year-olds and five-year-olds ready for the crucial entry test.

At present, the citywide gifted programs are accepting only students who score at the 99th percentile or higher! The more demand, the fewer places and the higher the cutoff score.

Black and brown students are nearly 70 percent of the public school enrollment, but win only 27 percent of the seats in gifted programs. So much for Bloomberg’s plan to expand opportunities!

To understand the nightmare that Bloomberg and Klein foisted on the city’s children, read Josh Greenman’s recent account of his family’s experience. Josh is on the editorial board of The New York Daily News, which is very pro-charter and pro-testing.

He writes:

How does the process work? Four-year-olds take a nationally normed standardized test (actually, two tests, the NNAT and the OLSAT, which are supposed to measure reasoning ability and general intellectual aptitude). No bubble sheets: It’s administered in person by an adult. Those above 90th percentile qualify for district programs. Those above 97th percentile qualify for citywide programs.

Those are the technical qualification thresholds. In practice, you need a 99 to qualify for a citywide school and usually something like a 95 to qualify for a districtwide program, though it depends on the district.

Once you get in the door as a kindergartener, you stay in the school or program through fifth grade (in the case of district programs) or eighth or 12th (in the case of citywide schools).

If this strikes you as kind of nuts, well, that’s because it is: A test taken on one day as a 4-year-old, a test for which your parents can prepare you, can put you on one track, separate and apart from your peers, for your whole K-12 education.

The citywide schools are coveted. They have excellent reputations and are by most objective measures very good schools. Of course they’d be, as the kids only get in through an intense filter, essentially ensuring engaged parents and high test scores.

They also, surprise surprise, have few black and Latino students and fewer low-income kids than the citywide average…

Why the hell should kindergarteners, first graders, second graders and so on have separate programs in district schools, much less separate citywide schools? Isn’t this part of a big underlying problem, letting (mostly) whites opt out of the common public system?

It’s a very fair question…

Would we consider it a victory if eliminating those programs resulted in a public school system that’s now 70% black and Latino 80% or 90% black and Latino?

Of course, that outcome depends upon what individual parents do, including how they respond to having their kids, who they often consider advanced, taught in general education classrooms.

But my head hurts when I start to think through how unfair the process is, at least in New York City, for plucking young kids out of general-ed classrooms. I’m also cognizant of how doing that intensifies racial and ethnic and income segregation, and related resentments. And of the negative effect of draining a small number of “chosen” kids, who tend to have intensely engaged parents with extra time and money on their hands from those classrooms.

Josh’s daughter made it into a local G&T program. He recognizes the trade offs. He understands that the G&T programs keep white and Asian families in the city and the public schools.

But that was not the rationale in 2007. The rationale was that having a standardized test with a citywide cut score, the same in every district, would expand opportunities for black and Hispanic students. Bloomberg and Klein said that tightening the admissions requirements would increase diversity! Anyone familiar with education policy and practice could have told Bloomberg and Klein that a single high standard on standardized tests would have a dramatically negative effect on children of color. At the time, they tried to tell them. But they were arrogant and they never listened to anyone outside their corporate MBA (masters of business administration) circle.

Here is a parent who warned them in 2007 that basing admissions to the gifted programs would be a disaster and would increase segregation and decrease opportunity for the children who need it most.

Bloomberg was a great mayor on matters involving public health and the environment.

But on education, he surrounded himself with businessmen and corporate types, and he took their bad advice about the virtues of high-stakes testing, standardization, privatization, letter grades for schools, and “creative disruption.” Bloomberg should not be boasting to the NAACP now about his non-existent accomplishments. And the NAACP should not listen to Bloomberg, no matter how much money he offers them.

 

Mike Deshotels, veteran educator, exposes the myth of high standards in Louisiana in this post. 

He discovered that John White, the State Superintendent of Education, has systematically and secretly lowered the state standards to make it appear that the state was making progress every year.

The raw scores on Louisiana’s state tests are kept secret from the public and the legislature. Deshotels got them by making public records requests backed up by 4 successful lawsuits that he won against John White for withholding public records. 

All anyone ever sees are the scale scores which seem to be stable, but the underlying raw scores change depending on what the LDOE wants them to show. So, White has now inflated the state test scores compared to NAEP by an average of 59% in just a few years.
As a result of his lawsuits, this is what Deshotels found. 
“Basically the Department of Education was allowed to set any standard they chose relative to the percentage of questions answered correctly. And they were also allowed to change that underlying percentage for passing without consultation from year to year. The passing standard has been quietly watered down over a period of years without the public or the legislature being informed. So at the end of the 2017-2018 school year my public records requests revealed that a student on average only needs to get about 30% of the questions right on their math and English tests in order to get a passing score. That’s just a little above what a student who knows absolutely nothing could attain with outright guessing….
”Even though 20% of students are repeatedly failing their state tests, public records reveal that only 1.8% of 4th and 8th graders are denied promotion. The truth is that the Louisiana Department of Education, using the latest BESE policy, expects our local school systems to promote basically all students to the next grade each year whether they have learned the material or not. Then the teachers in the next grade are magically supposed to teach them the new material in addition to what they did not learn in previous grades…
”As this blog explained in an earlier post, the improved graduation rate of Louisiana students is achieved using even more of the John White standards magic. Using the secret raw score standards implemented by John White, a student can pass his/her algebra I test by scoring only 15% correct answers. Geometry requires only 12% correct answers. English I can be passed by getting 17% of the questions right. Louisiana’s improved graduation rate was achieved by faking the stats….
“The National Assessment of Education Progress (NAEP) is a national test that is considered the gold standard for measuring proficiency of students in 4th and 8th grade in reading and math. On the latest NAEP test given, only 26% of Louisiana 4th graders achieved a proficient rating in reading, only 27% of Louisiana 4th graders got a proficient rating in math, only 25% of Louisiana 8th graders got a proficient rating in reading, and only 19% of Louisiana students got a proficient rating in math. My analysis reveals that our state tests have been inflated an average of 59% in recent years compared to the NAEP tests…

”The latest NAEP test results which compare Louisiana student performance in reading and math to all other states places Louisiana at its lowest ranking ever. We now rank at the bottom of all state systems. The only area scoring lower on NAEP is Washington D.C.

“Don’t blame the students or the teachers. The fact is the Common Common core standards are so bad, so age inappropriate, so filled with stuff these kids will never use, that the tests should not be used for any purpose, much less the promotion and graduation of students. Meanwhile our students are being denied instruction in real world problems and truly useful reading and writing skills.”

Shocking as this is, John White may have learned this trick while he was working for the Bloomberg-Klein regime in New York City, where the same thing happened on the state tests. The State Education Department watered the passing standards down every year from 2006-09, and it magically appeared that there was steady, even dramatic progress. The scoring on the tests was changed so that the number of students who scored a 1 (the lowest) fell to the lowest number ever. Bloomberg was able to boast about the “New York City Miracle” during his 2009 re-election campaign. The miracle disappeared after he was re-elected, after the State Board of Regents brought in outside experts to review the results, and after the scoring was recalibrated. At the time, the chair of the State Board of Regents was Mayor Bloomberg’s good friend, billionaire Merryl Tisch.
You can read the story in my book, “The Death and Life of the Great American School System,” pp.-78-79.
Here is the short version. The state began annual testing in 2006, and every year from 2006-09, the state made it easier to pass. “In 2006, significant numbers of New York City Students scored at level 1 and were subject to retention. The number of students at level 1 dropped so low that level 1 could hardly be considered a performance level. In 2006, 70,090 students in grades three through eight were at level 1 in mathematics; by 2009, that number had fallen to 14,305. In reading, the number of level 1 students fell from 46,085 to 11,755…In sixth-grade reading, 10.1 percent were at level 1 in 2006, but by 2009, only 0.2 percent were.”
Students in level 1 were denied promotion and entitled to remediation. Most were bumped from level 1 to level 2 by lowering the standards, thus allowing them to advance but denying them the remediation they needed.
The standards dropped so low that many students could reach level 2 by guessing.
A neat trick so long as no one notices.

Bill de Blasio was elected Mayor of New York City in 2013. He appointed former Deputy Chancellor Carmen Farina as his chancellor. Having been a teacher, a principal, and a superintendent, she was far more qualified than her predecessors Joel Klein (a lawyer) and Dennis Walcott (a non-educator who had served as Mayor Michael Bloomberg’s Deputy for education), and teachers welcomed a leader who had knowledge and e perience. However, Farina never cleaned house. People hired by Joel Klein remained in top positions. Farina, of course, had worked for Klein too.

The new Chancellor Richard Carranza has done some shaking up, although the deep strata of corporate reformers remain sheltered in place throughout the upper echelons of the Department of Education.

Leonie Haimson reports here on the resignation of one top Klein hire after two recent demotions and the retention of others.

Back in the days of Mayor Bloomberg and Joel Klein, the policy of the city was to close big high schools that had varied programs (e.g., music, the arts, advanced classes in math and science) and replace them with small schools. Almost every large high school in the city was closed. One of them was DeWitt Clinton, which had become a dumping ground for the small schools that did not want students with low test scores, English learners, and students with disabilities. In effect, the school–once known for its excellence–was turned into a graveyard.

Klein and his acolytes touted the New York City Miracle, built on testing, testing, testing, and small schools.

A piece of the architecture fell apart recently at DeWitt Clinton, when teachers leaked that students who never attended class were getting good grades and graduating, based on credit recovery by computer (at home).

This high school is a hooky player’s dream.

At DeWitt Clinton HS in the Bronx, kids who have cut class all semester can still snag a 65 passing grade — and course credit — if they complete a quickie “mastery packet.”

Insisting that students can pass “regardless of absence,” Principal Pierre Orbe has ordered English, science, social studies and math teachers to give “make up” work to hundreds of kids who didn’t show up or failed the courses, whistleblowers said.

“This is crazy!” a teacher told The Post. “A student can pass without going to class!”

The 1,200-student Clinton HS is one of 78 struggling schools in Mayor deBlasio’s “Renewal” program. Last year, 50 percent of seniors graduated, but only 28 percent of the grads had test scores high enough to enroll at CUNY without remedial help.

The DOE’s academic-policy guide says students “may not be denied credit based on lack of seat time alone.” Passing must be based “primarily on how well students master the subject matter.”

Orbe has taken the policy to a absurd extreme, teachers charge.

Leonie Haimson of Class Size Matters says that DeWitt Clinton, which is part of Mayor de Blasio’s “Renewal Schools” initiative, has very large classes, some as large as 39. That’s one remedy that the mayor ignored.

By the way, if you want to meet Leonie (and me), we will be at the annual dinner of Class Size Matters tomorrow night in New York City. It is not too late to get a ticket.

Eight years ago, I wrote a book about corporate reform and pointed out that the deliberate killing of large high schools had eliminated specialized and very successful programs for students, including advanced classes in math and science, and programs in the arts.

Today, the New York Times observed (too late to matter, too late to save Jamaica High Schoool in Queens or Christpher Columbus in the Bronx) that the Bloomberg-Klein decision to close large high schools and replace them with small schools has effectively destroyed successful music programs. The compensation is supposed to be that the graduation rate is higher in the small schools. But as I reported in my book, “The Death and Life of the Great American School System: How Testing and Choice Are Undermining Education,” the small schools enroll different students from the large schools they replaced. The neediest students are shuffled off elsewhere.

The Times reports today, in a long article,

“When Carmen Laboy taught music at Christopher Columbus High School in the Bronx, beginning in 1985, there were three concert bands. The pep band blasted “Malagueña” and Sousa marches on the sidelines at basketball games, and floated down Morris Park Avenue during the Columbus Day parade. The jazz band entertained crowds at the Ninth Avenue Food Festival, and even warmed the room at a Citizens Budget Commission awards dinner at the Waldorf Astoria.

“Today, Columbus no longer exists. In its former building, which now houses five small high schools, a music teacher struggles to fill a single fledgling concert band. Working out of Ms. Laboy’s old band room in the basement, Steven Oquendo recruits students for a sole period of band class from his school, Pelham Preparatory Academy, and the others on campus, with their different bell schedules and conflicting academic priorities.

“It does make it much more difficult to teach,” he said. “But we always find a way of making it happen.”

“Between 2002 and 2013, New York City closed 69 high schools, most of them large schools with thousands of students, and in their place opened new, smaller schools. Academically, these new schools inarguably serve students better. In 2009, the year before the city began closing Columbus, the school had a graduation rate of 37 percent. In 2017, the five small schools that occupy its former campus had a cumulative graduation rate of 81 percent.

“But one downside of the new, small schools is that it is much harder for them to offer specialized programs, whether advanced classes, sports teams, or art or music classes, than it was for the large schools that they replaced. In the case of music, a robust program requires a large student body, and the money that comes with it, to offer a sequence of classes that allows students to progress from level to level, ultimately playing in a large ensemble where they will learn a challenging repertoire and get a taste of what it would be like to play in college or professionally.

“In a large concert band, “you’re not the only trumpet player sitting there — there’s seven of you,” said Maria Schwab, a teacher at Public School 84 in Astoria, Queens, who is also a judge at festivals organized by the New York State School Music Association. “And you’re not the only clarinetist, but there’s a contingent of 10. In that large group, there’s a lot of repertoire open to you that would not be open to smaller bands.”

“The new schools chancellor, Richard A. Carranza, himself a mariachi musician, has said that he plans to focus on the arts, which can especially benefit low-income or socioeconomically disadvantaged students, according to the National Endowment of the Arts. A 2012 analysis of longitudinal studies found that eighth graders who had been involved in the arts had higher test scores in science and writing than their counterparts, while high school students who earned arts credits had higher overall G.P.A.s and were far more likely to graduate and attend college.

“The Bronx offers an illustration of how far Mr. Carranza has to go. There, 23 high schools were closed during the Bloomberg era, second only to Brooklyn. Of 59 small schools on 12 campuses that formerly housed large, comprehensive high schools, today only 18 have a full-time music teacher. In many of those, the only classes offered were music survey courses known as general music, or instruction in piano or guitar, or computer classes where students learn music production software. Only eight schools had concert bands, and of those, only five had both beginner and intermediate levels.”

The students with cognitive disabilities are not in the new small schools. The English language learners, the newcomers who speak no English, are gone.

Schools that once enrolled 4,000 students now house five schools, each with an enrollment of 500 or less. Do the math. When you disappear 1500 of 4,000 students, it does wonders for your graduation rate!

You can deduce this from the article, but it is never spelled out plainly. The small schools are not enrolling the same students as the so-called “failing high schools” of 4,000. The subhead of the article reads: “Downside of Replacing City’s Big Failing Schools.” I suggest that the big high schools were not “failing.” They were enrolling every student who arrived at their door, without regard to language or disability.

This is not success. This is a deliberate culling of students that involves collateral damage, not only the shuffling off of the neediest students, but the deliberate killing of the arts, advanced classes, sports, and the very concept of comprehensive high school, all to be able to boast about higher graduation rates for those who survived. A PR trick.

 

When Bill de Blasio ran for Mayor the first time, he sought my help. We met and spoke candidly. He told me he would strongly support traditional public schools. He said he would oppose the expansion of private charters into public school space. He promised to stop closing schools because of their test scores. His own children went to public schools. He would protect them and end the destructive tactics of Joel Klein, who coldly and cruelly closed schools over the tearful objections of students, parents, and teachers.

I enthusuastically endorsed him. The campaign issued a press release. De Blasio was elected in 2013, and re-elected in 2017. I wanted him to succeed and to support public schools against the privatizers.

He tried to stand up to the charters, but Eva’s billionaire backers rolled out a multi-million dollar TV campaign and donated huge sums to Governor Cuomo and key legislators. That ended de Blasio’s effort to block charter expansion. The legislature gave them a blank check in New York City, allowed them to expand at will, and even required the city to pay their rent in private facilities if it couldn’t provide suitable public space. Now his majority appointees to the city board rubber stamp charter co-locations and expansions.

Although the Mayor and Chancellor Farina have tried to support struggling schools, they have not hesitated to close them when they don’t show test score gains.

At the last meeting of the city’s Board of Education (which Mayor Bloomberg capriciously named the Panel on Education Policy to indicate its insignificance in the new era of mayoral control but which is still called the Board of Education in statute), the Mayor submitted a list of schools to close. Sadly, like Bloomberg, he has closed many schools. Unlike Bloomberg, he does not boast about it. There’s that.

At the last meeting of the Board, onee of the Mayor’s appointees, T. Elzora Cleveland, dissented and another abstained, denying the majority needed to close two of the schools on the Mayor’s list. Cleveland has resigned, and education activists assume she was forced out to make way for a more pliable board member. 

How is this different from Mayor Bloomberg’s tactics?

During the Bloomberg regime, the Mayor ousted three appointees who objected to his wish to end social promotion. The three members worried that no one had devised a plan to help the kids held back. Bloomberg fired them on the spot, and said, in effect, mayoral control means I am in charge and my appointees do as I wish. At the time, the firings were called “the Monday night massacre.”

I strongly oppose closing public schools, especially those that are historic anchors of their community. Several years back, I was on a panel with John Jackson, president of the Schott Foundation for Public Education. He said he had traveled to many countries to learn how they dealt with struggling schools. In every country, the Minister of Education said, “If a school is struggling, we send in support.” Dr. Jackson asked, “What do you do if you send support, and the school doesn’t improve?” In every case, the Minister said, “We send in more support.”

The bottom line is that accountability lies with the leadership. If a school is in trouble, it is up to the leadership to help, not punish. They control the resources. They decide whether the school will reduce class sizes and have the staff and programs it needs. Accountability begins at the top.

 

Gary Rubinstein admits that he misses the big names of reform whose stars have flickered out: Michelle Rhee, Joel Klein, Arne Duncan, Cami Anderson, and those others whose words could be picked apart and ridiculed.

Gary says the successors to the golden oldies are not nearly as much fun. He explains by quoting at length from the current leader of Teach for America, whose prose is flat, bland, and blah. She even quotes George W. Bush on the “soft bigotry of low expectations.” How low can you go? Well, maybe some day she will quote Donald J. Trump to inspire the troops.

He writes:

“The disappearance of the reform rock stars and replacement by this new breed of bland understudies was a first step in the collapse of the reform movement. Trump and DeVos surely have not helped Democrats continue to embrace ‘school choice’ as a viable solution. Then, you knew it had to happen eventually, Bill Gates recently came out and admitted that teacher evaluation reform didn’t work as well as he had predicted so he is going to instead work on curriculum development. Whether or not the reform movement is merely ‘playing possum’ right now and playing dead while really planning their next wave of attack (some are giddy about the upcoming Janus Supreme Court case), I suppose we will find out in the years to come.”