Archives for category: Testing

William Mathis, managing director of the National Education Policy Center, wrote this post for the Blog.

“Pay no attention to that man behind the curtain!”

A Closer Look at the Changing School Privatization Claims


​Flashed on a 1939 version of a jumbotron, the great and mighty Wizard of Oz appears, wreathed in great billows of green smoke, as a reverberating announcement commands Dorothy, the Tin man, the Scarecrow and the Cowardly Lion to “Pay no attention to that man behind the curtain!” Of course, as we all know, the mighty wizard was all pretense, magnification, and illusion flanked by great balls of fire.

​And so it has been. The privatization wizards have issued countless press releases saying that charter schools In Boston, Florida, Chicago or almost anywhere else are a great success because of standardized test score increases.

For years this omnipresent claim is that turning schools over to outside contractors will result in grand progress as measured by great leaps – in standardized test scores. All we have to do is follow them down the yellow brick road.

Indeed, Chris Lubienski and Jameson Brewer note in a NEPC review, “a whole generation of school reforms has elevated test scores as the predominant metric by which to judge the worth of policies, as well as of schools, teachers, and even in some cases subjection of public schools to choice regimes through federal policies like No Child Left Behind (NCLB).” (1)

Yet, lately, things have not been so rosy in Oz or for the school privatization wizards. Several recent, large scale and well-designed studies have concluded that privatization has not produced the mighty test score gains promised.

Yet, Toto persisted in chewing on the curtain and ERIC, IES and a host of other scholarly and well-respected organizations concluded there really wasn’t much difference in test scores between public and non-public schools. In fact, in places such as Washington, DC, Indiana, and Louisiana, statewide evaluations have shown no advantage and, ominously, some have found actual test score losses as a result of privatization reforms. (2)

​Since their primary argument doesn’t look so good, the reformers now say, “Don’t look behind the curtain! Instead, look over there at attainment.” Attainment is the new goal which is a potpourri of indicators such as graduation rates, higher education attendance, higher education graduation, absenteeism and the like. These are certainly worthy goals which would be embraced by most people. Now, the pro-privatization purposes and measures are being shifting away from testing. In a complete about face, they ask, “Do impacts on Test Scores Even Matter?” (3)

​Lubienski and Brewer address this “Don’t look here, look over there” shift-the-goal phenomenon in a recent NEPC think-tank review of an American Enterprise Institute paper presented at the Association for Education Finance and Policy’s annual spring conference. While the study has not been peer reviewed, it was provided with booming publicity by charter advocates. (4)

​Fordham’s Michael Petrilli, a prominent advocate for test-based reform, shows remarkable agility (perhaps realizing that the test score results were not very impressive), by concluding that “focusing on test scores may lead authorities to favor the wrong school choice programs. It’s a legitimate concern, and one I share…the experience of attending a private school in the nation’s capital could bring benefits that might not show up until years later: exposure to a new peer group that holds higher expectations in terms of college-going and the like; access to a network of families that opens up opportunities; a religious education that provides meaning, perhaps a stronger grounding in both purpose and character, and that leads to personal growth.”

​Buttressing this maudlin appeal to national pride, religion and personal growth, Petrelli shuffles the studies to get a different result and says, “yes, impacts on test scores matter” and urges caution in making too much of research literature that comes to a contrary conclusion.

​Robin Lake joins the shift saying, “We now believe effectiveness must be considered more broadly, as preparing children with the knowledge, skills, and analytical capacities necessary for them to navigate the new realities of an information economy and be able to prepare for rapid changes in workforce demand.” (6) The shift from mechanistic hard test scores has the reformees saying “look over there!’ (7)

​What’s missing is that we’re more in Kansas than in Oz. This is not Dorothy waking from a bad dream proclaiming “there’s no place like home.” It is a bad reality as many children have no home and society provides Dorothy and her classmates with only ersatz opportunities and facile shifts of words, phrases and promises rather than the reality of good schools for all.

Endnotes:

[1] Lubienski, C. & Brewer, T. J. (2018). Review of “Do Impacts on Test Scores Even Matter? Lessons from Long-Run Outcomes in School Choice Research: Attainment Versus Achievement Impacts and Rethinking How to Evaluate School Choice Programs” Boulder, CO: National Education Policy Center.

[2] Barnum, M. (July 12, 2017). Do School vouchers “work.” As the debate heats up, here’s what research really says. Chalkbeat. Retrieved April 30, 2018 from https://www.chalkbeat.org/posts/us/2017/07/12/do-school-vouchers-work-as-the-debate-heats-up-heres-what-research-really-says/
Dynarski, M. (2016, May 26). On Negative Effects of Vouchers. Brookings: https://www.brookings.edu/research/on-negative-effects-of-vouchers/
Turner, C, & Kamenetz. (June 26, 2017) School Vouchers Get 2 New Report Cards. NPR https://www.npr.org/sections/ed/2017/06/26/533192616/school-vouchers-get-a-new-report-card
Spector, C. (February 28,2017). Vouchers do not improve student achievement, Stanford researcher finds. https://news.stanford.edu/2017/02/28/vouchers-not-improve-student-achievement-stanford-researcher-finds/

[3] Hitt, C., McShane, M., & Wolf, P. (2018) Do impacts on test scores even matter? Lessons from long-run outcomes in school choice research: Attainment versus achievement impacts and rethinking how to evaluate school choice programs. Washington, D.C.: American Enterprise Institute. http://www.aei.org/wp-content/uploads/2018/03/Do-Impacts-on-Test-Scores-Even-Matter.pdf

[4] Lubienski, C. & Brewer, T. J. (2018). Review of “Do Impacts on Test Scores Even Matter? Lessons from Long-Run Outcomes in School Choice Research: Attainment Versus Achievement Impacts and Rethinking How to Evaluate School Choice Programs” Boulder, CO: National Education Policy Center.

[5] Petrilli, M, (April 17, 2018). For the vast majority of school choice studies, short- and long-term impacts point in the same direction. Fordham Institute. https://edexcellence.net/articles/for-the-vast-majority-of-school-choice-studies-short-and-long-term-impacts-point-in-the

[6] Lake, R. (May 1, 2018). How Can We Get Serious About Successful Pathways for Every Student? Center on Reinventing Public Education.

[7] Saultz, A. et al.(April 2018).Charter School Deserts: High-Poverty Neighborhoods with Limited Educational Options. Fordham Institute. Retrieved May 1, 2018 from https://edexcellence.net/publications/charter-school-deserts-report

The Associated Press reviewed the Gates Foundation’s education spending and found that Bill Gates is now engaged in “helping” shape the ESSA plans of the states. He just can’t stop telling everyone how to run their public schools even though everything he has tried until now has been a failure.

Does he care that teachers in several states have walked out due to underfunding? Is he trying to persuade governors and legislators to tax billionaires to raise school funding? Don’t hold your breath.

Presumably he wants to make sure they are sticking with high-stakes testing, Common Core, and charters, his three favorite reforms to which he never subjected his own children or their school. If it is not good enough for Lakeside Academy in Seattle, why must it be mandatory for the rest of the nation?

Louisiana has gone crazy for testing.

Mercedes Schneider reports that the state plans to spend at least $75 million for five years of in PARCC-like tests. Testing will begin in kindergarten.

Louisiana to Spend at Least $75M on Five Years of PARCC-ish LEAP 2025 Testing

The state’s scores were abysmal on the latest NAEP, and apparently State Superintendent John White and the state board believe that if they weigh the pig more often it will get fatter.

This is insane.

Tennessee has had three straight years of computer breakdowns and other problems with its testing vendor, Questar. (Questar is also the test provider for New York State, perhaps others.)

The state is so disgusted by Questar’s performance that Governor Haslam said he might switch the contract from Questar to ETS.

That’s a good one! As testing guru Fred Smith informed me, ETS owns Questar.

ETS is not problem free, either. “The changes highlight a possible strategic shift for ETS whose reputation came under fire last year when the nonprofit had to pay $20.7 million dollars in damages and upgrades after multiple testing problems in Texas.”

Would someone in Tennessee please let Governor Haslam and Commissioner McQueen know?

Questar is the for-profit face of nonprofit ETS. Tweedle Dum and Tweedle Dee.

A distinction without a difference.

[Testing expert Fred Smith points out that Questar is owned by ETS. Much ado about nothing.]

Tennessee Governor Haslam says the state may drop Questar, its exam vendor, because of three straight years of problems with the state tests.

The contract may go to ETS, he said.

One positive note in the testing failure is that data won’t be available to assign low-scoring schools to the failed and ineffectual “Achievement School District.”

Tennessee likes to boast about how it moved from the cellar on NAEP to the middle, but the article cited here was written before the release of NAEP 2017, where scores in the state mostly declined compared to 2015.

John Merrow recently served as a judge for the Education Writers Association’s annual reporting awards. While admiring the high quality of journalism that he read, he used his post to excoriate Arne Duncan and Margaret Spellings for a self-serving opinion piece that they wrote in The Washington Post.

“Here’s the story that shouldn’t be ignored: The proponents of disastrous ‘school reform,’ which has given us 20+ years of ‘test and punish’ & such, are now positioning themselves as voices of common sense. Exhibit A is this recent Washington Post column by two former Secretaries of Education, Arne Duncan and Margaret Spellings. One guided the Department under George W. Bush’s “No Child Left Behind,” and other created the infamous “Race to the Top” program.

“Their breath-taking chutzpah begins with the title of the piece: What ails education? ‘An absence of vision, a failure of will and politics.’ But their opening sentence actually tops it: “We have long benefited from a broad coalition that has advanced bold action to improve America’s education system.”

“Just exactly who are the WE that have benefited from the ‘bold action’ that the Secretaries refer to? It’s far easier to identify those who have NOT benefited from “No Child Left Behind” and “Race to the Top.” Let’s start with students, because their performance on the National Assessment of Educational Progress, which everyone agrees is education’s ‘gold standard,’ has basically been flat for the 20+ years of Bush and Obama. Next on the list are teachers, whose salaries and morale have declined over the years of increasing reliance on multiple-choice testing and ‘test-and-punish’ policies. Collateral damage has been done to the occupation of teaching, which has lost prestige and now fails to attract enough candidates to fill our classrooms with qualified instructors.

“So that’s–literally–millions of students and teachers who have NOT benefited from the ‘broad coalition’ that Duncan and Spellings are so proud of.”

Who benefitted from the Duncan-Spellings billions and mandates?

Testing corporations. Ideologues who want to fracture public education. Profiteers. “And–surprise–the two former United States Secretaries of Education. One now leads the University of North Carolina higher education system, and the other is one of three Managing Partners of The Emerson Collective, Laurene Powell Jobs’ very wealthy and active education venture.”

Wow.

He then goes on to enumerate the “reformers” who are now backpedaling or mansplaining, all to avoid responsibility for the disasters of the past 20 years. They (including Duncan and Spellings) are the people we need to be reduced from, says Merrow.

This is one of Merrow’s best pieces. He is on a roll.

As the jaws of destruction and disruption approach to privatize their schools, teachers and parents at many schools in Puerto Rico have organized to boycott the standardized tests that will be used to rationalize the closing of their schools.

Jesse Hagopian tells the story here, after interviewing Mercedes Martinez, president of the Teachers Federation of Puerto Rico.

Teachers in Puerto Rico, she told him, were inspired by the historic strike against MAP testing at Garfield High School in Seattle, which Jesse helped organize.

“Another part of the disaster capitalist approach to schooling has been to impose the same high-stakes standardized testing regime that we have in the mainland U.S. on the Puerto Rican education system. High-stakes testing is being used to punish schools, students, and teachers. With teachers living in fear of the consequences low scores, they are forced to teach to the test, not the student, and it is causing a narrowing of the curriculum to what the corporate test makers believe is important to learn (For example, I don’t think the lessons of how to organize your community against corporate education reform will be on the next test). These tests are used as exit exams in high school and are denying thousands of students the chance to graduate. Perhaps worst of all, the testocracy has trained people to believe that wisdom is the ability to eliminate wrong answer choices on a multiple choice exam, rather than to be creative, empathize, or solve real life problems.

“But a mighty movement of parents, students, and teachers has risen up to boycott and opt out of these tests as a way to reclaim public education and fight for authentic forms of assessment.”

 

Assessment experts Judith Singer and Henry Braun wrote an article for Science warning about the risks of misusing the results of international tests. The article is behind a paywall, and I don’t have a subscription. Singer was interviewed by a writer for The 74, and she expressed her concerns. The bottom line is that the rankings distort more than they reveal.

“The rankings that are commonly used to report the results of [international tests] draw headlines, but they are often incredibly misleading,” she told The 74. “The countries aren’t sports teams to be ranked as winners and losers.” Indeed, she observed, the British press uses the same term to describe the hierarchy of international testing performance — “league tables” — as for soccer and rugby standings.”…

“Worse than the alarmism accompanying news stories, Singer says the rankings themselves are frequently arbitrary and mercurial. Positions change from year to year for reasons having little or nothing to do with student performance in a given country. And the rules of the tests allow for a certain amount of gamesmanship, as when Shanghai earned a top ranking for math in the 2012 PISA exam — only for the world to later discover that it had excluded 27 percent of its 15-year-olds from taking it.

”On the 2015 PISA, Japan improved on its fourth-place ranking for scientific literacy three years earlier, moving to second overall. But the jump wasn’t because of improved performance; scores actually went down, though not as much as other countries’.

“In a Japanese news item on the results, a graph shows scores and rankings over time. A line representing the country’s science ranking ascends from 2012 to 2015 — even though actual scores dropped by nine points.

“Deep-seated national differences also tend to skew our perceptions of who’s up and who’s down. It doesn’t really make sense, Singer remarked, to group countries with decentralized education sectors — like the United States, Canada, and Germany — alongside those with properly national school systems, such as France, that can mandate instructional and curricular choices at will across their entire student populations…

““Singapore has fewer schools than Massachusetts has school districts,” Singer said. “So when you look at the results of Singapore — which is a city-state, though it’s treated as a country — you’re talking about a very small jurisdiction. There are undoubtedly school districts in Massachusetts that far exceed the performance of Singapore.”

“Drawing apples-to-apples comparisons among disparate countries with wildly varying educational approaches leads to false narratives about what produces success, with low-performers looking to emulate the “special sauce” driving high achievement — whether it’s special curricula, smaller class sizes, or something else — in high-flying countries like Finland or Korea.”

“Rather than spending millions trying to ape the tactics of international competitors, Singer says that countries should use testing data to learn more about themselves.“

 

 

In this post, John Merrow asks and answers the question,

“If YOU had the power to make ONE major change in American public education immediately, what would you choose to do?”

“I posed the question to my dinner companions, three authors and one editor.  But before I tell you how they answered the question, please take a minute to decide what you would do.”

The United States has required every child in grades 3-8 to take standardized tests in math and reading every year since NCLB was signed into law in 2002. No high-performing nation does this. Typically, they test children once in elementary school, once in middle school, once in high school. Finland, recently designated “the happiest nation in the world” and also high-performing, has no standardized tests in grades 3-8. Teachers write their own tests and are tested to grade them.

Chris Churchill is a columnist for the Albany Times-Union.

Churchill: For better schools, ditch the standardized tests

It’s easy to think of things our kids would be better off doing. Playing in the spring sunshine. Planting a garden. Burying their heads in books. Practicing jump shots. Catching frogs. Learning reading, writing and arithmetic. Learning Urdu. Learning anything.

 The tests are a time suck for teachers, too. They’ll be watching over spiritless and possibly anxious classrooms of test-taking students when they should be, crazy thought here, teaching. We should want our schools alive — with passion and joy, with laughter and curiosity, with play and learning.

Maybe that sounds too romantic for this grim, hard-headed age. Shouldn’t we insist that our children line up for the rat race and defeat their rivals from around the globalized economy?

Even if we swallow that baloney, there’s remarkably little evidence that the national rise of high-stakes standardized testing has done anything to improve schools and learning. As far as I can tell, the only beneficiaries are the big bureaucracies that want more control over classrooms and the big corporations that provide the tests.

The tests certainly haven’t benefited our kids, who, in many districts, are getting shorter recesses so teachers can spend more time in service to the looming tests. Or who, as many parents can attest, view testing days with anxiety and dread.

If the tests were just tests, they might be relatively harmless. But they epitomize something bigger: The madness that applies a production mentality to education. Everything can be neatly quantified, yes siree, not to mention automated, regulated and homogenized!

But children aren’t widgets and schools aren’t factories. You can’t measure the success of a classroom with data points. Standardized testing tells us nothing important about how children experience school.

Tests can’t tell us if Mr. Jones is a much-needed role model for fatherless boys. They can’t tell us how much Mrs. Riley cares for her fourth-graders. They can’t tell us if Ms. Hughes’ eighth-graders feel supported or inspired. They can’t tell us if Mr. Hernandez is changing lives.

All of which illustrates why tying teacher evaluations (and salaries) to test scores is so hideously ludicrous. Such a system rewards an uninspired teacher who devotes every depressing classroom minute to dreary test prep, and it punishes the impassioned teacher who refuses to teach for the test but instead imbues children with a love of learning.

There are other problems. Tests designed by upper-middle-class professionals will, surprise surprise, inherently reward the children of upper-middle-class professionals. Schools attended by poor kids get labeled underperforming or even failing. But lower test scores often result from that very poverty. A child who knows violence, hunger or fear at home won’t do as well on a standardized test, and it’s unfair to expect even a great teacher to overcome that.

Let’s pause here to give the opt-out movement a sincere and robust round of applause. Clap, clap, clap, clap.

The parents who hold their children out of testing — about 20 percent of the statewide total in recent years — are expressing healthy rebellion against the production approach to education. They’re standing up to the consultants and “experts” who claim to know what’s best for kids but prove again and again they don’t. They are saying no to an impersonal education bureaucracy with a vested interest in getting bigger and silencing parental voices.

Clearly, the opt-out movement has been a tremendous success. It has forced New York to back off its testing regime, at least a little. The time devoted to testing students in grades 3 through 8, for example, has been reduced from six days annually to four, including the two days of math testing that begin Tuesday.

Meanwhile, Gov. Andrew Cuomo has seemingly shelved his proposal that test scores account for 50 percent of teacher evaluations; the opt-out rebellion put that bad idea on ice. Now, the state Assembly is even considering a bill that would end test-based teacher evaluations altogether.

New York should go further. It should altogether eliminate standardized testing in elementary and middle schools.

Doing so would be a step toward rejecting the insidious idea that education should be evermore standardized. It would bring more local control of schools. It would help recognize what should be obvious: Real teaching can’t be homogenized, because every child learns differently. It’s an inherently individualized process.

As most every parent and teacher knows, learning is about small moments and quiet victories. It’s about relationships built on trust and even love. My God, is there anything more personal or magical or maybe even divine than teaching a small child to read?

There are things that can be measured. Teaching and caring for children are not among them.

cchurchill@timesunion.com • 518-454-5442 • @chris_churchill