Archives for category: Testing

Many readers have contacted me to ask why CNN has not posted Randi Kaye’s interview with me, rebutting Michelle Rhee’s assertions.

This reader, Michael Brocoum, made a copy of the interview and posted it on Youtube. Here it is.

As I mentioned in an earlier post, the interview began with a question about the National Assessment of Educational Progress (you will note that it is misspelled by CNN as the National Assessment of Educational Process). I don’t recall the precise wording, but the question went like this:

“You claim that test scores on NAEP are at their highest point in history, but how do you explain that the scale score for fourth grade reading is only 221? That’s 221 out of 500. That’s less than 50%. Isn’t 50% a failing grade?”

I then tried to explain that scale scores don’t work like that, that the question itself was a completely erroneous interpretation of scale scores. NAEP has a vertical scale, and scale scores in the 4th grade are lower than in the 8th grade. They can’t be converted into a grade in the way that Randi Kaye asserted, although they are useful as measures of progress.

Consider this: the average scale score for 4th grade is 221, but students scoring at the 90th percentile–our top students–have a scale score of 264. By Randi Kaye’s fallacious reasoning, they are failing too! In 8th grade reading, the students at the 90th percentile had a scale score of 307 (on a scale of 500). She would convert that to a grade of 61, which is borderline failing.

Wouldn’t you think that the editor or research staff at CNN would have prevented Randi Kaye from making such absurd assertions?

But it was of a piece with all the questions that followed. I felt as if I were being interrogated by someone who worked for StudentsFirst, not by a reporter seeking to ascertain either my views or the basic facts.

Rhee said at the outset of my interview that the answer to what she thinks is the terrible performance of our schools is merit pay. So Randi Kaye drilled in on that with two questions (one of them was dropped from the show before it aired). She ended up with a quote from someone named Lucas who said he wanted merit pay. That wasn’t exactly definitive, since I was able point out that merit pay has been tried again and again and has always failed to make a difference.

I have often been struck by the uneven playing field that policymakers and legislators establish for charter schools and public schools. The public schools are increasingly strangled by regulations and by high-stakes testing and punitive evaluations, at the same time that the charter schools are exempt from most of the strangulation. I have heard many times from principals who say that they want to turn their public school into a charter so they can escape the tentacles of regulation that are wrapped tight around their school. And I have wondered whether the purpose of “reform” was to make public schools fail while the deregulated charter schools increase and thrive.

Here is another take on the current corporate reform movement, inspired by an earlier post about stagnant ACT scores:

The more conversations I have about the entire “reform” movement, the more convinced I am that it’s really about disbanding teacher unions so that the majority of education programs will eventually be part of a private industry thus paving the way for the privatizing of all public systems.

The evidence just keeps mounting to show that standardized testing is a flawed way to judge the efficacy of the public schools, and the mere fact that charters and private/parochial schools do not have the same “rigorous” standards as public schools points to the idea that “standards” are not really important at all to the reformers who push for these kinds of alternate schools.

Utilizing standardized tests that the reformers know are flawed is a tactic to devalue the people who teach in public schools so that they can be fired and a private interest can take over.

It’s as if these policy-makers have found a way to rig the game: Create new rules that make for impossible goals and then watch a good system that serves the public fail under these new rules. They have set up the game so that the players will fail no matter what–IF you believe the rules are sound.

It’s pretty evident that the main goal is to disband two of the largest public unions in the country using children as pawns. Once the AFT and the NEA are toppled, so they must think, the rest will follow, and the privatization of public systems in America will ensue.

I would not put it past our policy-makers to be trying to sell the public a bill of goods by pretending they care about the children at all, when in reality, they care about getting rid of union teachers and privatizing education so their buddies can “invest” and continually get rich.

This isn’t about parent choice (unless legal segregation is what they’re after), this isn’t about success, and this isn’t about getting rid of “bad” teachers. It’s about getting rid of unions and privatizing. To me, the evidence points to these intentions no matter how anyone else wants to spin it.

A small group of public high schools in New York City managed to get exempted from the testing regime of the New York Regents many years ago.

And they have proven themselves.

These schools use performance assessments rather than the standardized tests of the Regents (although they do take the Regents exam in English language arts). The performance assessments are demanding. They are judged by teachers, parents, and others.

The students in these schools are succeeding at far higher levels than the students in other public high schools in the city. They have a higher graduation rate, a lower dropout rate, and a higher rate of persistence in college than students from other public high schools in the city. They have higher graduation rates for black students, Hispanic students, Asian students, white students, and students with disabilities than the New York City public schools in general.

Now they want to increase their number from 28 (26 in the city) by adding another 19.

But city and state officials are reluctant. What if they let other schools escape the testing regime? Who knows how many would seek to be exempt next year and the year after. What if no one were left to be tested? What do results matter, as compared to the sense of control and power that high-level officials thrive on?

If test scores are the measure of education (and I don’t think they are or that they should be), then our present course of “reform” is a bust.

This is what FAIRTest had to say about the ACT scores, released today:                                                                                                                                                                                                                                                                             

 

STAGNANT ACT RESULTS, GROWING RACIAL SCORE GAPS
MORE EVIDENCE THAT TEST-BASED SCHOOL “REFORM” IS ON WRONG TRACK;
“WHY ARE POLICY-MAKERS DOUBLING DOWN ON A FAILED STRATEGY?”

        Flat ACT college admissions exam scores released today show, “The nation’s decade-long fixation on high-stakes testing as the primary tool to boost academic performance and narrow learning gaps is a sweeping, expensive failure,” according to FairTest, the National Center for Fair & Open Testing. ACT results, which claim to measure college readiness, are unchanged over the past five years. The test score gap between racial groups has risen slightly in the same period, according to ACT data.

“Rational policy-makers would look at the evidence and change course,” said FairTest public education director Bob Schaeffer. “Yet, instead of abandoning what is clearly the wrong track for improving U.S. schools, policy-makers are actually putting more weight on standardized tests.” Recent U.S. Department of Education waivers of the controversial “No Child Left Behind” (NCLB) law require states to use tests for more high-stakes purposes, including evaluating teachers. Many states have also increased standardized exam requirements.”

Schaeffer, concluded, “Why are policy-makers doubling down on a failed strategy? How much more data do they need to understand this approach is not working?”

Reports by the National Research Council of the National Academy of Sciences and other experts have also concluded that high-stakes testing programs have not raised student achievement. Results from the National Assessment of Educational Progress (NAEP) confirm that academic improvement has slowed under NCLB.

– – 3 0 – –

 2012 COLLEGE BOUND SENIORS AVERAGE ACT SCORES
Approximately 1.66 million test takers

                                                     COMPOSITE SCORE     FIVE-YEAR SCORE TREND
                                                                    (2008 – 2012)

ALL TEST-TAKERS                                     21.1                                 0.0

Asian                                                        23.6                              + 0.7

White                                                        22.4                              + 0.3

African-American                                       17.0                              + 0.1

American Indian                                         18.4                             –  0.6

Hispanic                                                    18.9                              + 0.2

source:  ACT, The Condition of College & Career Readiness 2012

Robert D. Shepherd has been in the education publishing industry for many years. When I was writing The Language Police a decade ago, Shepherd was a reliable guide to the vagaries of the publishing world. I also found him to be an acute observer of language and literature. I am happy he wrote this to share with you:

I would like to point the would-be reformers of American education to the work of that great political and social theorist Robbie Burns, who wrote in “To a Mouse” that “The best-laid schemes o’ mice an’ men / Gang aft agley.”

A bit of old-fashioned Scots skepticism with regard to this latest attempt at centralized planning of education, the Common Core State [sic] Standards, is in order. If history is any guide (and what other guide do we have?), the latest top-down reform efforts will fail miserably, and for predictable reasons.

The theory behind the latest “reform” efforts comes to use from the business community. In 1992, Robert Kaplan and David Norton published an article in the Harvard Business Review called “The Balanced Scorecard—Measures That Drive Performance.” Kaplan and Norton picked up on and refined a business truism—that you get what you measure and reward—and gave it a new spin: You shouldn’t rely simply on financial measures, which are backward-looking, but, rather, should create key performance indicators (KPIs) in four areas—finance, customer satisfaction, processes, and knowledge, and follow those carefully. The article set off a revolution in American business. Suddenly, everyone was talking analytics and performance measures and employee evaluation based on those, and it was only a matter of time until business people and politicians, ever thick as mosquitoes over a swamp, got together to apply the same reasoning to education. The theory was simple: Create standards and hold people accountable for achieving them. Thus NCLB was born. The Common Core State [sic] Standards can be thought of as NCLB v2.

So, what could be wrong with holding people (teachers, administrators, students) accountable to standards? As is so often true, the devil is in the details. If you read closely the supporting documentation coming from the CCSSO, Achieve, and the two testing consortia, you will find that in English Language Arts, the intent of the new standards is to make texts, and responding to texts, primary. The whole point is to produce students who, upon graduation from high school, can read, understand, and respond to college-level materials. The standards themselves, however, are simply lists of skills and concepts to be mastered. Since teachers’ and administrators’ jobs will be on the line, they will be incentivized to do everything in their power to make sure that students are working down the lists, mastering standard RL.1.1, then standard RL1.2, and so on. However much the standards-touting organizations attempt to communicate that there is a difference between standards and curricula, the whole apparatus of assessment and data-crunching will focus on the standards themselves, in isolation. How are African-American female students doing on standard SL.3.2a, according to the tests? In other words, states and local districts will inevitably treat the standards as curricula. We are already seeing, across the nation, online curriculum development tools cropping up for use by districts in their curriculum planning, and these tools inevitably begin, at the top of the page, with the standard to be covered.

In the old days, a teacher of 11th-grade American literature would do a unit on the American transcendentalists, in which students would read the likes of Emerson, Thoreau, and Dickinson. Teachers and students would focus on the ideas and texts of the transcendentalists—self reliance, communion with nature, the Oversoul, etc., and in the course of reading, discussing, and writing about these authors and their stimulating ideas, students would learn some concepts and skills. That’s as it should be. People’s brains are networks, connection machines, and new learning occurs when that learning is attached to an existing semantic network. You take a class in oil painting at a local community center. In the course of a week, you learn what gesso, a filbert brush, stippling, and chiaroscuro are. And the new learning sticks with you because it is connected in an experiential network that is meaningful to you. If, on the other hand, you tried to memorize a telephone book, you would mostly likely fail because brains are not good mechanisms for learning facts, concepts, and skills in isolation.

Now, to their credit, the various standards-touting organizations are aware of this, and they have issued a number of documents, like the Publishers’ Criteria from the CCSSO, emphasizing that skills and concepts listed in the standards should not be taught in isolation, that students should deal with related texts, with texts in related knowledge domains, across a school year and across multiple years. However, the elephant in the room is the standards themselves, which are JUST lists of concepts and skills. In practice, the tendency will be to force teachers into scripted work in which they know that it is November 28th because they are “doing,” today, standard RI.5.7. Just today I received in the mail a catalog from a textbook publisher containing its new Common Core State Standards offerings—workbooks that “do” one standard at a time, in order, treating the standards themselves as a curriculum.
So, that’s the first problem. Standards are not a curriculum, but in practice, that’s how they will be treated.

The first reason why the new standards regimen is likely to fail has to do with how people learn: they learn in semantically connected contexts that they care about in which the content is primary.
The second reason why the new standards regimen is likely to fail has to do with how people work. Let’s go back to business management theory for a moment. There is a body of theory in management called Social-Technical Systems Theory, the basic premise of which ought to be obvious: almost everyone wants to do a good job, to be able to be proud of what he or she does, to have his or her work recognized, and in order for that to happen, people have to have autonomy. Theodore Roosevelt put it this way (and here I am paraphrasing): If you want to get something done, find someone who knows how to do it and get the hell out of that person’s way. In other words, good managers don’t micromanage. They specify goals, but they don’t specify how the work is to be done. Suppose that you hire someone to clean your house or apartment and then stand over that person and tell him or her how to pour the water and cleaning solution, how to move the mop, and so on. Chances are that however much you are paying for this work, that person will not return again, for you have violated a fundamental law of human nature: we all HATE to be micromanaged because we value our freedom and autonomy. The blueprint for the new ESEA (the Elementary and Secondary Education Act) calls for 50 % of the evaluation of every teacher and administrator in a district receiving federal funds being based on improvement in test scores. So, inevitably, with their jobs on the line and fancy electronic data-collection systems in place, administrators will micromanage classrooms. Today is November 28th. You and your students need to be following this script so that the students can master standard W.3.2a for the test.
It’s not the INTENTION of the best of the standards makers to have teachers treat the standards as curricula or to have their work be micromanaged and scripted, but inevitably, that’s what will happen, and inevitably, a few years down the line, we shall see the new reform that throws out the old reform and starts all over again, promising another miracle cure for what ails the country’s education system. The best-laid plans of mice and men go often astray.

There’s one more problem that I would like to mention. The whole idea of a top-down, standardized education system is incompatible with fundamental principles of liberty and pluralism. There are no standardized teachers. There are no standardized students. And there shouldn’t be. No one’s five-year plan will work, and if it did, God help us. We wouldn’t end up with the diversity that we need. We need, very much, to get out of teachers’ way, to let them do what they, idiosyncratically, do. Let tens of thousands of flowers bloom. If you think back on the best teachers that you ever had, you will inevitably find that not one of them was following a script. Instead, their interests, and what you learned from them, were highly idiosyncratic. This person was PASSIONATE about Beowulf or analytic geometry, and you caught the windfall of that person’s passion. You got the bug. And that’s what we need in a pluralistic society, not robot students coming out of schools-as-factories, identically machined to have the same concept and skill sets, but, rather, the bustling, blooming variety of interests, inclinations, passions, and abilities that a complex contemporary society requires—some who are passionately interested in graphic design, some who are passionately interested in equity trading, some who are passionately interested in prenatal development, and so on. You can’t get rich diversity of skill sets from ANY list, however well vetted.

I regularly read the posts from Phil Cullen in Australia, which he calls “The Treehorn Express.” Here is one of his best.

You  will notice that he includes a link to Valerie Strauss’s Answer Sheet, showing how ideas travel fast around the world.

NAPLAN is the Australian national assessment program of literacy and numeracy.

The Treehorn Express
 
 
Scores, Scores, Scores
The delusional, paranoiac obsession with scores and numbers in educational dialogue, and their use by educators who should know better, constantly diverts attention from the real issues. We have all been guilty of using statements like:- “Finland has the best system in the world, because it topped the PISA tests in 2009.” “USA has a poor system of schooling. It ran 27th on the PISA tests.”  “Australia and New Zealand are in the top ten of the best school systems in the world.” “Asia’s cram schools are raising the stakes.”  What is this system of ranking that has led us to the use of such inanities?
PISA is a Programme for International Student Assessment, operating under the auspices of OECD. It tests only 15 year-olds [considered to be school leavers] in a number of countries to see how well-equipped they might be to face the world at large. Although the PISA is only able to test the testables in reading, mathematics and problem solving, it has no link to any school curriculum and provides great fun for the measurement nerds at OECD, Paris. It is claimed to be “…a powerful tool to shape government’s policy making.”   Heaven knows why. Thus far, it has created chaos and panic amongst those in countries who don’t understand what it is.  It tested reading in 2000; mathematics in 2003; science in 2006; reading again in 2009. For 2012 some 15 year-olds are being randomly selected from about 30 countries to test mathematics and try an optional computer-based assessment of mathematics and reading. PISA carries more punch than it deserves. For curriculum use and for comparative standards, its punch would not explode a paper bag.
It is influential, however. Countries, states and authorities around the world have gone numbers-mad to copy its impetuous ardour:- giving tests invented by local non-school measurers, assigning numbers as scores to each participant, averaging the numbers to declare some ridiculously impossible assessments of teachers, principals, schools and systems; publishing results as if they carried some sort of evaluation of what was going on in regard to teacher competencies, school performance, principals’ curriculum leadership and systems’ organisation.
In Australia, these unreliable, comparative number-scores are used as the basis for serious but totally inaccurate descriptions of pupils, schools, of teachers, of principals; have given rise to an amazing array of gimmickry; enhanced the coffers of private schooling; and enriched the coffers of publishers. Nothing much else. They have been used to describe ‘best’ schools and ‘worst’ schools, ‘good’ teachers and ‘bad’ teachers. Some newspapers have been cruel, with commentators pontificating on standards of schooling. the needs for this and that, They have even been used to describe countries as providing outstanding educational services because of success in this test, that is unfamiliar to most commentators and has yet to be extensively examined as a reliable device for what it says it does.  The Australian Gratten Institute, founded in the same year as NAPLAN, established to advise governments on policy matters, contributes to the heresy by its reliance on numbers to make judgements. It’s report: “Catching Up: Learning from the Best School Systems in East Asia” where after-school tutoring to raise test scores is rife, bases it’s contents on numbers scored. [http://www.saveourschools.com.au ] “The report is seriously deficient and one-sided.” says Trevor Cobbald.   One hopes that policy-makers will consult with humanity-biased commentators and the education community before any serious decisions are taken as a consequence of this report.
And all this malarchy costs over $540million with more to come to prop-up the [officially] failed NAPLAN testing scheme!
The reliance and over-use of Arabic numerals for educational purposes is catastrophic. Measurers are people who dwell on the outskirts of educational activities and who  greatly exaggerate the power of number scores. They should get back in their box with their childish toys.  Number is number. When its hieroglyphics are used for descriptive purposes, scores and marks and numbers are inappropriate. Used for serious evaluation of education’s human effort as they are during the present testing pandemic, the use is satanic.
saynotonaplansaynotonaplansaynotonaplansaynotonaplansaynotonaplansaynotonaplansaynotonaplansaynotonaplansaynotonaplandaynotonaplansaynotonaplan
        [Kelvin Smythe’s criticism of a NZ Shadow Minister’s statement contains some brilliant summaries of the effects of ‘national standards. The Minister then responds.]IMPORTANT READING:     http://www.washingtonpost.com/blogs/answer-sheet/post/eight-problems-with-common-core-standards/2012/08/12/821b300a-e4e7-11e1-8f62-58260e3940_blog.html#pagebreak
                                          [Marion Brady comments: “Future historians…are going to shake their heads in disbelief. They’ll wonder how, in a single generation…democracy has dismantled its engine.”]

A group called  EdVoice filed a lawsuit to compel the teachers of Los Angeles to comply with 40-year-old legislation (the Stull Act), requiring that student performance be part of teacher evaluation. The Los Angeles teachers’ union has opposed the suit, and they are currently in litigation.

The issues are supposed to be resolved in December. There are many ways to demonstrate pupil performance, not just standardized test scores.

There are so many of these “reform” groups that it is hard to keep track of them.

What and who is EdVoice?

Here is an answer from Sharon Higgins, an Oakland parent activist who is a relentless researcher, and who maintains two websites, one called “charterschoolscandals,” the other called “The Broad Report.” Higgins is an example of the power of one voice, devoted to facts:

Here are a few more specifics about EdVoice, the education lobbying group.

EdVoice was founded in 2001 by Reed Hastings (CEO of Netflix, Microsoft board member, Green Dot founding funder) and John Doerr (venture capitalist, investment banker), along with and former CA state Assembly members Ted Lempert and Steve Poizner. Eli Broad and Don Fisher (deceased CEO of The Gap and major KIPP supporter) once served on EdVoice’s board.

EdVoice has received a ton of money from all of the above as well as from Carrie Walton Penner and Fisher’s widow, Doris. Penner lives in the Bay Area and is a Walton Family Foundation trustee. She also sits on KIPP’s board, as does Reed Hastings, and the Fishers’ son, John.

Back in 1998, Hastings also co-founded Californians for Public School Excellence with Don Shalvey. This is the organization that pushed for the Charter Schools Act of 1998, the law that lifted the cap on the number of charter schools in the state.

Don Shalvey was involved with starting the first charter school in California, just after the passage of the California Charter School Act of 1992 (CA was the second state to pass a law). He is also founder and former CEO of Aspire Public Schools. Reed Hastings has been a major source of Aspire’s financial backing, including its launch. In 2009, Shalvey stepped down from his post at Aspire and went to work for the Gates Foundation, but for a while he stayed on Aspire’s board. The Gates Foundation has given generously to Aspire.

In 2011, Hastings and Doerr pumped $11M into DreamBox Learning, an online education company started by a former Microsoft executive and the CEO of a software company. It was acquired by Hastings with help from the Charter School Growth Fund.

BTW, EdVoice co-founder Lempert is currently president of an Oakland-based org called Children Now; he occasionally teaches at Cal. Poizner, a conservative Republican and wealthy Silicon Valley high tech entrepreneur, was defeated by Meg Whitman in the June 2010 gubernatorial primary, and is now the State Insurance Commissioner. For several years he worked for Boston Consulting Group as a management consultant.

More conniving fun and games.

A reader recalls what Neil Postman wrote almost 40 years ago. He was known as a future-thinker, and this quotation proves he was:

I’m a public school teacher in NJ. Relevant in this context is a wonderful little book by Neil Postman entitled “The Disappearance of Childhood”. He ties the invention of childhood in the west over the past few centuries to universal literacy. In the book he argues that the concept of childhood peaked sometime in the first half of the 20th century, but since then has been worn down by a decline in literacy and our infatuation with other forms of entertainment media. Profiting from schooling, racing to the top, focusing on absurd objective criteria like standardized test scores might be further evidence of a declining belief in the idea of childhood, where children are something more like competitive employees than learners.

In his 1976 book “Crazy Talk, Stupid Talk” (now there is a title for our times!) Mr. Postman referred specifically to the trend toward objectivity in evaluating children:

‘In many communities around the country, the quality of education is being measured by the scores children achieve on standardized tests. Education thus falls under the jurisdiction of the language of statistics, and it is a fact that many schools are now designing their programs almost solely for the purpose of increasing their students’ mean test scores. Here, it is slightly unfair for me to repeat the joke about the statistician who drowned while trying to wade across a river with an average depth of two feet. The fault is not with statisticians, whose special language is a remarkably useful instrument for uncovering abstract facts. The fault is with those educators who have fallen under its spell and have allowed their purposes to be subverted by the seductions of precise measurement.’

Jeff Bryant reflects on the events and ideas that transformed American society in the past century.

A current exhibition at the Museum of Modern Art in New York City describes the 20th century as “the century of the child.”

Now with technocratic ideas dominant in national policy, championed by both Obama and Romney and semi-articulated by Arne Duncan, Jeff wonders if this is “the century of the coldhearted.”

In California, there is a battle going on between a “reformist” group called Edvoice and the United Teachers of Los Angeles. California has a forty-year-old law called the Stull Act, which says that a teacher’s evaluation will include evidence of student progress. But how is “progress” defined?

Some “reformers” will use any and all opportunities to make standardized test scores the measure of teacher quality, because that guarantees that 1/2 of all teachers will be below the mean and thus many will be “ineffective” by design. Knowingly or not, they seem eager to create evidence of failure.

This principal says that “progress” means far more than bubble-test scores:

To willingly comply with policy that one knows is heinous, is nothing short of injurious to both individual teachers, teachers’ unions, and students. I have been a teacher and principal in California for my entire educational career (30+ years) and can tell you with certainty that VAM based on California’s Stull Act (AB 293) is a complete perversion of that bill’s purpose, and most importantly, practice.
I am not a lawyer, but I do know that past practice and precedent count for a lot in legal circles and California’s Stull Act has never been interpreted to mean that teachers must be evaluated on a student’s standardized test scores. The bill simply states that teachers will be evaluated on “student progress”, but since when is a student’s progress limited to a standardized test score? To go along with VAM based on standardized test scores is to make an assumption that the sum and substance of a student’s school performance and experience is limited to the bubbles he or she is filling in on any given day: an absurdity. Since 1971 when the Stull Act went into effect, “student progress” was something that was interpreted by the teacher being evaluated and the school principal. Yes, the evaluation could include test scores, but just as valid were criterion referenced tests, publishers tests, student portfolios, teacher observations, principal observations and, as of late, even the observations of other teachers or bargaining unit members. The important thing was the agreement between the teacher and principal about what was appropriate considering the teacher’s stated goals.
To indicate that UTLA should “go along” with despicable policy is a symptom of exactly why teachers across the nation have found themselves in a predicament like they’ve never before experienced. Standing up against policy that is just plain wrong has nothing to do with hiding anything. Indeed, going along with such policy as VAM shows a willingness to comply with atrocious policy, even though you know that it is injurious to both students and teachers, an act that doesn’t garner the respect of anyone. It’s time to draw the “line in the sand” and do what we know as professionals is the right thing: resist VAM and encourage our unions to fight it at every opportunity.