Archives for category: Testing

Scantron, the test publishing company, was compelled to delete a reading passage that was highly propagandistic after parent activists learned about it and called attention to it. The item was brought to the attention of the media by Parents United for Responsible Education.

The Chicago Sun-Time wrote: “PURE executive director Julie Woestehoff said the passage, titled “Reforming Education: Charter Schooling,’’ is so one-sidedly pro-charter that its use amounts to an attempt to “brainwash” children ‘with propaganda about charter schools.’'” Julie Woesterhoff is a co-founder of the national parent organization Parents Across America.

The reading passage on the test was a paean to charter schools, with pie charts and bullet points, all intended to show that charters were decidedly superior to the public schools in which the test-taking students were enrolled. It even had the nerve to identify a presumably fictional “multimillionaire” who enrolled his own children in a charter school. It would be interesting to know if there are any real-life multi-millionaires who have done so. I guess that the folks who wrote the test passage didn’t know that charters are supposed to be “saving” poor kids from failing schools, although not many of them do that.

The test question was presented as “non-fiction,” but Scantron initially responded by saying it was fiction intended to test reading comprehension. Even Scantron eventually realized that the question was inappropriate. That is putting it mildly. The question was charter propaganda, intended to misinform students and persuade them that charters were proven better than public schools. That’s not inappropriate, that’s lies.

It may not be coincidental, but it’s worth noting that Scantron was a corporate sponsor of ALEC. When the publicity about ALEC’s role in the Trayvon Martin affair got too hot, Scantron was one of the corporations that withdrew from ALEC.

The fake charters-are-best question is an even bigger scandal than Pearson’s pineapple question. The pineapple story (which by the way was given to Illinois students in the past) was at worst idiotic, not insidious. It was in some way typical of the sanitized, vacuous reading passages that often appear on standardized tests, which explains how it got past the test review panels that approve test content.

The charter question is far worse than the pineapple question. The pineapple question wasn’t selling pineapples. It was not an advertisement for Dole or another corporation. The charter question was taking a one-sided stance on a matter of public policy. It was dishonest propaganda. It advanced a political cause and, in today’s reality, it advanced the commercial interests of for-profit charter operators.

Do these people have no shame?

Diane

Everyone interested in understanding how the ceaseless pressure to raise test scores can corrupt the tests should be familiar with Campbell’s Law.

This is an adage written by social scientist Donald T. Campbell in a 1976 paper. It says:

“The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.” (You can google the paper, or find it linked on Wikipedia: Campbell, Donald T., Assessing the Impact of Planned Social Change The Public Affairs Center, Dartmouth College, Hanover New Hampshire, USA. December, 1976.)

Campbell’s Law explains why high-stakes testing promotes cheating, narrowing the curriculum, teaching to the test, and other negative behaviors..

In his 1976 paper, Campbell also wrote that  “achievement tests may well be valuable indicators of general school achievement under conditions of normal teaching aimed at general competence. But when test scores become the goal of the teaching process, they both lose their value as indicators of educational status and distort the educational process in undesirable ways. (Similar biases of course surround the use of objective tests in courses or as entrance examinations.)”

Campbell’s Law helps us understand why No Child Left Behind and Race to the Top are harmful to education. They put pressure on teachers and principals and school districts and states to get higher and higher test scores. As we saw in Atlanta and in Washington, D.C., this kind of pressure may cause educators to betray their ethical duty by changing test scores. As we saw in New York state, this kind of pressure may cause a state education department to lower the passing mark on state tests so as to boost proficiency rates.

As high-stakes testing has become the main driver of our nation’s education policy, we will see more cheating, more narrowing of the curriculum, more gaming of the system. None of this produces better education. And even the test scores–on which so much public policy now firmly rests–will be corrupted, by making them so important.

 Diane

Mitt Romney is out on the campaign trail, pushing vouchers and charters and online learning and for-profit schools and larger class size as the answers to our “failing” public schools.

I wish someone would give him some actual facts to work with. Are our schools failing? No, they are  not.

According to the latest federal data, the high school graduation rate is now at the highest point in our history for every group: for white students, black students, Hispanic students, low-income students, middle-income students, and high-income students.

According to the National Assessment of Education Progress, test scores in reading and math are at their highest point in our history. Forgive me if I quote an earlier blog from this site:

“Proficient [on NAEP] is akin to a solid A. In reading, the proportion who were proficient in fourth grade reading rose from 29% in 1992 to 34% in 2011. The proportion proficient in eighth grade also rose from 29% to 34% in those years. In math, the proportion in fourth grade who were proficient rose from 18% to 40% in the past twenty years, an absolutely astonishing improvement. In eighth grade, the proportion proficient in math went from 21% in 1992 to an amazing 35% in 2011.”

“When the scores are broken out by race, you can really see dramatic progress, especially in math. In 1992, 80% of black students in fourth grade were below basic. By 2011, that proportion had dropped to 49%. Among white students in fourth grade math, the proportion below basic fell in that time period from 40% to only 16%.”

“The changes in reading scores are not as dramatic as in math, but they are nonetheless impressive. In fourth grade, the proportion of black students who were below basic in 1992 was 68%; by 2011, it was down to 51%. In eighth grade, the proportion of black students who were reading below basic was 55%; that had fallen to 41% by 2011.”

These numbers tell a story not of failing schools, but of steady–and in some cases, very impressive–progress.

Should we do better? Of course. But people don’t do a better job if you keep telling them (falsely) that they are failing. It is important to acknowledge success if you want to keep moving forward.

Mitt Romney tried pushing his education policies at a charter school in West Philadelphia. He probably thought that what he was offering would be greeted with cheers, but he looked very foolish when he told his audience that class size didn’t matter.

Steven Morris, a music teacher at the school, said: “I can’t think of any teacher in the whole time I’ve been teaching, over 10 years — 13 years — who would say that more students would benefit them. And I can’t think of a parent that would say ‘I would like my kid to be in a room with a lot of kids,’” Morris said. “So I’m kind of wondering where this research comes from.”

Romney knew better than the teacher, it seems, because he cited a study by McKinsey saying that class size doesn’t matter. No doubt, he also had heard the same from his stable of uber-conservative think tank experts.

Had Romney consulted a wider body of research, he would have known that class size does matter.

Had he thought about the choices he made for his own children, he would have not been so foolish as to suggest that class size doesn’t matter. I don’t know where they went to school, but I have read that they were educated in private schools. I am willing to bet they had class sizes of 12-18. (A reader informs me–see comments below–that the Romney children attended an elite school where average class size was 12. Wonder how that would work in the public schools of Detroit, Cleveland, Fresno, Philadelphia, and Baltimore?)

Why would Romney propose that children who need as much or more attention as his own children should get less?

Diane

A parent in Texas wrote to say that she couldn’t understand why the state was paying Pearson $100 million a year while laying off teachers. She’s right. This is crazy. She pointed out that in addition to the direct cost of the state testing, schools and districts now had to pay people whose sole job is the care and feeding of the testing monster. One district is hiring a testing coordinator for each of its five high schools, More money diverted from the classroom. At the same time the cost of testing grows, the budget for public education shrinks.

She sent me this article from an Austin newspaper: http://www.kxan.com/dpp/news/local/austin/educators-parents-fight-testing-system. Sandy Kress, who was the architect of NCLB and is now a lobbyist for Pearson, strongly defended the testing system, saying that young people would be closed out of good jobs if they didn’t take all those tests.

Now, be it noted that this claim is utterly false. Students in independent schools (such as the one that Kress’ own children attend) do not take all those tests and they presumably will not be shut out of the good jobs in the future. http://jasonstanford.org/2012/05/the-lone-staar-rebellion/

Furthermore, there is no reason to assert that taking state tests prepares anyone for good jobs in the future. Where is the logical connection? How does testing prepare you to get a better job? The testing regime now in force penalizes students who exhibit imagination or divergent thinking. Entire generations of Americans have gotten good jobs without being subjected to test prep and annual high-stakes testing.

According to the Bureau of Labor Statistics (http://www.bls.gov/opub/mlr/2009/11/art5full.pdf, p. 88), most new jobs will not require a college degree.

And where is the evidence that taking all those state tests is the best way to prepare for college? Again, none of the children who attend elite independent schools take those tests and they seem to have a high rate of success in gaining admission to selective colleges and universities.

Really, the test salespeople and lobbyists for the testing industry have sold the American people a bill of goods. Either we buy their product, and more of it, and pay them for prep materials, and pay them for test security, and pay test coordinators, or no one will get a good job in the future.

Don’t believe it.

Diane

Chicago Superintendent of Schools J.C. Brizard has admitted that he does not know how to improve Chicago’s public schools. He did so by asking the Gates Foundation to supply millions of dollars to open another 100 charter schools. http://www.chicagotribune.com/news/local/ct-met-cps-charter-growth-20120517,0,7306759.story

Handing public schools over to private management is a frank admission of failure on the part of school leadership. It amounts to saying, “I don’t know how to improve them. Let’s turn the kids over to the private sector and see if they can do it.”

Of course, Brizard arrived with an uneven record after having served briefly as superintendent of schools in Rochester. While he was there, the low graduation rate fell even lower. And, even though he claimed test scores gains, the proficiency rate in English and math was less than 30% when he left for Chicago. Brizard claimed that the state had raised standards in 2010, but in fact the state education department admitted that it had dropped the passing mark on state tests year after year, creating a false image of progress. http://www.chicagonewscoop.org/jean-claude-brizard-report-card-uneven-performance-in-short-rochester-tenure/

It is a shame that Mayor Rahm Emanuel was unable to find a superintendent for the city’s public schools that was able to develop a plan to improve them. Not having a plan, Brizard is ready to throw in the towel and privatize them. But, then, as a graduate of the Broad Superintendents’ Academy, he probably thinks that this is a good strategy, rather than an admission of defeat and failure.

Diane

I learned yesterday that the Colorado Department of Education has hired the ubiquitous firm Alvarez & Marsal to investigate a cheating scandal in Denver (http://www.ednewscolorado.org/2012/05/15/38261-state-investigating-two-denver-schools).

Alvarez & Marsal has no experience in investigating cheating scandals. The firm is a business consulting firm in New York City. It was hired a few weeks ago to investigate the Michelle Rhee cheating scandal in Washington, D.C. (by Rhee’s former deputy), but that investigation has not yet occurred. That scarcely constitutes expertise in this field.

A&M was hired to run the St. Louis schools several years ago. They sent in one of their principals, former CEO of Brooks Brothers clothing store, to run the district. After collecting $5 million, A&M departed and the district sank further into despair and was taken over by the state. A&M also did some consulting in New Orleans, which was awash in money for consultants.

Then A&M got a $16 million contract to rearrange the bus schedules in New York City. In its most memorable day, the firm was responsible for stranding thousands of children on street corners on the coldest day of the year. What New Yorkers remember best about A&M was that it charged the city $500 per hour for the work of its executives. Read that again, slowly. $500 an hour, not a day. The city’s Department of Education never apologized, and no one was ever held accountable for the fiasco. But as we now know, only teachers are held accountable, never the leaders of the school system or their highly paid advisors.

We will watch with interest to see what comes of A&M’s new role as the investigator of cheating on tests.

Diane

Aaron Pallas is one of the most insightful commentators on education in the nation. He teaches at Teachers College and if I were a student there, I would want to study with him. Not only is he smart, he is fearless. His regular columns in the Hechinger Report are “must” reading. His latest is about a teacher who was identified by New York City’s “teacher data reports” as the absolutely worst eighth grade teacher of math in the city. (http://eyeoned.org/content/the-worst-eighth-grade-math-teacher-in-new-york-city_326/).

Pallas shows how her rating has nothing to do with her performance as a teacher. She teaches gifted students and was a victim of her own success. Her students did so well last year that they did not meet the scores that the city’s computers predicted for them this year. But every one of them took and passed the state’s Regents exams.

The teacher, Carolyn Abbott, makes the interesting point in the article that the tests are high-stakes for the teacher, but not for the students. These students are so far advanced beyond the expectations of the state tests that the tests are almost a joke for them. But not for her. She is leaving teaching and going to work on a doctorate and then probably into college teaching.

Pallas writes, quoting Abbott: “I love to teach,” she says. And she loves mathematics. Ultimately, she decided, the mathematics was more important than the teaching, although she envisions teaching mathematics at the college level in the future. “It’s too hard to be a teacher in New York City,” she says. “Everything is stacked against you. You can’t just measure what teachers do and slap a number on it.”

When will the authorities in New York City and Albany and Washington, D.C., and in state education departments across the nation recognize that they have created a monstrous, counterproductive and utterly harmful means of evaluating teachers? Are they wise enough to recognize the errors of their ways?

Diane

The movement to slow down or stop or reverse high-stakes testing is moving forward at a rapid pace. This past week, the Houston Independent School Board endorsed a resolution opposing the overuse and misuse of standardized tests (http://blog.chron.com/k12zone/2012/05/hisd-joins-anti-testing-movement/). The resolution has now been endorsed by about 450 school boards in Texas, representing nearly half the state’s students.

The Texas resolution picked up steam after Robert Scott, the state commissioner of education, blasted the misuse of tests earlier this year. He said that testing had grown into the “be-all, end-all” of education and had become “the heart of the vampire.”(http://goo.gl/Az246)

Scott stepped down recently but it turns out that he spoke for vast numbers of Texans who are sick and tired of the tests that now control education and children’s lives.

Parents in Florida are now on board the anti-testing train, as are parents in New York.(http://online.wsj.com/article/SB10001424052702303505504577406603829668714.html?mod=WSJ_hp)

New parent organizations opposed to high-stakes testing seem to be forming in many cities and states.

I recently posed a question on Twitter that is relevant to this development. I asked, what happens in your district if children vomit while taking the test? I got many answers from teachers about the policy in their district. In some, the test must be placed in a plastic baggie and preserved. In others, the child must immediately retake the test. There were all sorts of variations on what to do when test anxiety causes a child to lose his or her breakfast.

Test anxiety is only part of the problem. Pineapplegate opened a national discussion about the quality of the tests and why they  are used to decide the fate of children and their teachers.

Diane

It turns about that Houston has been awarding test-based bonuses for years. It turns about that tying test scores to scores has not been good for teachers or students. It turns out that the ratings jump around from year to year. They are inaccurate, unreliable, and unstable. Value-added assessment, as everyone recognizes, creates massive pressure to raise scores on standardized tests of questionable value. The more pressure, the less reliable the scores. The more pressure, the more teaching to the test and the more cheating.  (http://nepc.colorado.edu/blog/houston-you-have-problem)

Value-added assessment is inherently incapable of producing better education because it does not measure better education. It only measures test scores. Higher test scores are a byproduct of better education. If you aim for the scores, you miss the target. The target is deeper understanding, greater knowledge, more thoughtful writing, more careful observation, a greater love of learning. The very act of measuring destroys the target instead of bringing it closer.

Governor Andrew Cuomo once boasted that he was the lobbyist for the children of New York state. One of his signal claims is the new evaluation system for educators.

Many educators think it is a very bad system because it relies heavily on standardized test scores. More than 1/3 of the principals of New York state signed a protest against the system and continue to fight it. They have said, repeatedly, that reliance on test scores for evaluations will harm their teachers and their students. Governor Cuomo doesn’t listen.

Now a new article was just published in Newsday by Mike McGill, the superintendent of the Scarsdale public schools, one of the best in the state. McGill asks why it is necessary to rank and rate teachers on a 100-point scale. As an experienced educator, he knows the downside of this measurement obsession.

Here is the article, with which I agree.

Diane

McGill: Rating won’t help teachers or kids

Published: May 11, 2012 12:40 PM
By MICHAEL MCGILL

$alttext

Photo credit: Illustration by Christopher Serra |

The State Education Department has mandated a new evaluation scheme for New York‘s teachers. In what Gov. Andrew M. Cuomo describes as a “groundbreaking” move, everyone now will be rated on a 100-point scale that relies heavily on classroom observations and students’ test scores.

What could be wrong with that?

Pretend you want to evaluate 150 people in your organization. You have three objectives. Assure a high level of effectiveness. Constantly improve everyone’s performance. Screen out anyone whose efforts aren’t acceptable.

You want to base your evaluation plan on several principles. You know that performance improves when people collaborate and when they get good coaching. Good coaches take information from multiple sources and use it to give considered feedback. Their charges get ample opportunity to practice under supervision. Evaluators also need standards and strong evidence to hold employees accountable.

Most of the people in your organization want to succeed. A few are truly exceptional. A larger number have mixed strengths and weaknesses. A smaller group is less competent. Everyone works independently much of the time. You don’t have the resources — enough supervisors or time, for example — to give everyone the continuous, thorough feedback needed to change complex behavior intentionally. So you focus your energies.

Some of your people are relatively new. They need more support and mentoring. You get several supervisors to collaborate in observing these newcomers and in working to bring them along. You’re less interested in comparing them than in whether each is becoming fully proficient — and then getting even better .

Some people are more expert. You check in on them less often to be sure they’re meeting core standards, cooperatively plan for their development, and offer them opportunities to hone their skills and absorb emerging knowledge about their field. Again, you’re less interested in how they might rank and more in their staying vibrant and continuing to grow.

You know from your periodic checks and from informal feedback that some folks aren’t measuring up. Supervisors either collaborate to help them upgrade their performance or develop extensive evidence for their dismissal.

That’s effective evaluation in a rational world. Not in the world of Albany. Albany wants to rank people relative to one another.

But why?

If the point is to help them improve, they need insightful advice and good coaching, not numerical rankings. If it’s to screen out less competent teachers, the only relevant yardstick is whether performance is up to standard. Who cares whether Ms. Jones is number 34, 35 or 36 out of 150?

The state’s rationale is that the metrics will drive people to compete for better scores. But what’s the point when the numbers lack meaning? Everyone knows that standardized tests aren’t good measures of who’s a good teacher, for example. Few, if any, researchers believe they can be used to make fine distinctions among practitioners, as the state plan tries to do.

Regardless, quantification is the name of today’s game. Student test results or classroom observations determine at least 71 points of a teacher’s score. The local schools control the remaining 29 points, but they have to be divided up in some set way: so many for planning, so many for taking part in professional activities, and so on.

This numbers game already drives teachers to spend increasing time prepping their kids for exams at the expense of other learning, and to play the system so they can amass points strategically. It’ll discourage collaboration, as well. As one veteran recently said, “Why should I do anything that could help someone else get a higher score than I do?”

Meanwhile, no rigid scoring formula will anticipate all possible situations. Let’s say Ms. Smith’s special needs kids are constantly the brunt of her dark sarcasms when nobody’s watching. That’s unacceptable. Whatever her strengths, credible student and parent feedback should lead supervisors to judge her performance inadequate. In Albany‘s 100-point world, however, she may well pile up enough points to be “proficient.” All she has to do is deliver a coherent lesson in front of an observer, produce decent test scores and strategically get a few more points here and there.

In short, the supposed strengths of this one-size-fits-all approach are really weaknesses. The “objective” numbers don’t judge people accurately. One state-wide evaluation framework doesn’t make sense for every school, and this one restricts the human judgment that’s essential to effective evaluation.

This is teacher appreciation week. In place of well-meaning sentiment, New York State should appreciate its teachers meaningfully. Rather than impose its uniform evaluation template on everyone, it should enable districts to develop their own plans and their capacity to evaluate effectively. A real service to teachers would be to help them understand whether teaching is the right career for them and, if it is, how to do an even better job of developing the determination, initiative, and thinking skills standardized tests can’t measure.

Michael McGill, superintendent of the Scarsdale Public Schools, is participating in a panel about the misperceptions and realities of the state’s teacher evaluation system on Saturday, May 12, at Bank Street College in Manhattan.