Archives for category: Data

On Anthony Cody’s new independent blog site, “Living in Dialogue,” Chicago teacher Michelle Gunderson offers her views on the ethical use of student data. 

 

In her many years as an elementary school teacher, she has seen standardized tests evolve from a sorting instrument to a means of punishing children to an excuse for privatizing public schools.

 

She will not be complicit in any of these uses of student test scores. She would abolish the standardized tests if she could, but that is not within her power.

 

So she pledges, first, that they will always be on of multiple measures; that she will remain strict confidentiality about student test scores and never publish them on a data wall or release them to the public; and that she will communicate with families about the frequency and amount of time spent on testing.

 

Tests, like all tools, may be used wisely or wrongly. Tests should be used to help children and teachers, not to punish or label them or close their school.

If you have not read Rachel Aviv’s “Wrong Answer” in The New Yorker, drop everything and read it now.

Aviv tells the story of the Atlanta cheating scandal through the ideas of one man, one teacher, who cared deeply about his student. Step by step, he got sucked into the data-driven obsession with test scores, thinking that if he raised the children’s test scores, it was a victimless crime. He knew that his students had needs that were even greater than their test scores, but the law’s absurd requirement that scores had to go up year after year drew him into a widespread conspiracy to falsify test scores.

One day will we look back on the Atlanta cheating scandal as the wake up call that made us think about how successive administrations and members of Congress have given their approval to laws and goals that hurt children and warped education? Or will we continue on the present path of destruction?

Marion Brady is a retired teacher and administrator and prolific author.

He writes:

“In a commentary in the July 21, 2014 issue of Time magazine, columnist Joe Klein takes aim at one of the usual targets of today’s education reformers—unions. In a dig at New York City mayor de Blasio, he says, “A mayor who actually cared about education would be seeking longer school days, longer school years, more charter schools…and the elimination of tenure and seniority rules…”

“Like just about every other mainstream media pundit, Klein thinks he knows enough about educating to diagnose its ills and prescribe a cure. That he’ll be taken seriously testifies to the power of what’s become the conventional wisdom, that if America’s schools aren’t performing as they should it’s because teachers aren’t getting the job done.

“What’s the teacher’s job? Raising standardized tests scores.

“What’s the key to high test scores? Rigor.

“What does rigor look like? No-excuses teachers doing their thing for as long as it takes to get the job done.

“What’s “their thing”? Teaching to demanding standards—the Common Core State Standards.

“The market-force-education-reform juggernaut set in motion by business leaders and politicians about a quarter-century ago is simple and easily summarized. (1) Adopt tough performance standards for school subjects. (2) Use high-stakes tests to measure performance. (3) Reward high-scorers; punish low scorers.

“Which, when you think about it, is off the mark. School subjects are just tools—means to an end. We don’t tell surgeons which scalpels and clamps to use; what we want to know is their kill/cure rate. We don’t check the toolbox of the plumber we’ve called to see if he (or she) brought a basin wrench and propane torch; we want to know that when the job’s done the stuff goes down when we flush. We don’t kick the tires of the airliner we’re about to board; we trust the judgment of the people on the flight deck.

“School subjects are tools. Kids show up for kindergarten enormously curious and creative. What we need to know is how well schooling is enhancing that curiosity and creativity. Kids learn an incredible amount on their own long before they walk through school doors. What we need to know is how much improvement there’s been in self-directed learning. Kids appear to begin life with an innate sense of what’s right and fair. What we need to know is how successfully that sense is being nurtured.

“We’re on a wrong track. Standards? Of course! But not standards for school subjects. What’s needed are standards for the qualities of mind, emotion, character, and spirit the young must be helped to develop if they’re to cope with the world they’re inheriting.
The Common Core Standards, says the CCSS website, “provide clear signposts along the way to the goal of college and career readiness.” Just stick to the CCSS script to be prepared for college and career.

“College? Years ago, the Association of American Colleges’s Project on Redefining the Meaning and Purpose of Baccalaureate Degrees said, “We do not believe that the road to a coherent education can be constructed from a set of required subjects or academic disciplines.” I’ve seen no evidence that the thoughtful among them have changed their minds.

“Careers? We have no idea how the interactions of globalization, automation, climate change, clashing societal
worldviews, and trends not yet evident will effect careers. The only thing that can be said with certainty is that nobody knows what careers are going to be available when today’s elementary school kids are looking for work.

“Back in the 70s, in his book Reflections on the Human Condition, Eric Hoffer, philosopher, writer, and longshoreman, wrote something that the Common Core Standards don’t adequately reflect: “In a time of drastic change it is the learners who inherit the future. The learned usually find themselves equipped to live in a world that no longer exists.”

“Standards? Sure. But not standards for solving quadratic equations, or for recalling the chemical formulas for salt, sand, baking soda, and chalk, or for interpreting Dr. King’s Letter from a Birmingham Jail as some self-appointed “expert” thinks it should be interpreted.
And not standards that make it easy to create machine-scored tests that perpetuate the destructive myth that quality can be quantified and turned into data to drive education reform.

“Standards—proper standards—could work wonders. Consider, for example, the effect just one standard could have on teachers, on teaching materials, on kids, on the citizenry, on America:

“Schools will be held accountable for sending learners on their way with a deep-seated love of learning and a willingness and ability to follow where that love leads.”

I did not write the following post. It was written by a high-level official at the New York City Department of Education who–for obvious reasons–requires anonymity. The story he tells is instructive. It is about how “reformers” claim victory by manipulating statistics. This is not an accusation directed at the de Blasio administration, but at their predecessors who regularly boasted that the new small high schools got better graduation rates that the large schools they replaced. The Gates Foundation bought this lie and has lauded its “success” in New York City.

 

 

 

 

Reformers Caught Lying. Again. This Time About Graduation Stats.

 

High school graduations are upon us. This is the time of year when parents, students, families, educators and communities celebrate the accomplishments of our high school seniors. It is a time to honor the work the graduates have done and to collectively share their hopes and dreams for the future.

 

At the same time, certain players in the world of education attempt to co-opt this time of year to propagandize for their favored reform policies. The latest example of this is a story about Frank McCourt High School, a small high school in New York City that “will send 97 percent of its first graduating class to college.” The story goes on to note that this school, along with 3 others, replaced a larger high school “which suffered from dismal academic and attendance records.” The reader is asked to believe one little and one big lie.

 

Let’s first take a look at the little lie. Does this school, in fact, have a 97% graduation/college going rate? The truth is that, on any reasonable calculation, it does not. According to the New York State data the cohort started with 100 freshmen. By sophomore year only 88 students remained. By junior year only 80. And by senior year only 69. Of these 69 survivors 67 are graduating. Seems more like a twisted version of the Hunger Games than a school for all students. One wonders: Where did the other 33 students go? Why does the media publicize such meaningless numbers without giving the full story? By now this trick should be well-known. A school that removes large numbers of students from its cohort should not be celebrated for its test scores or graduation rate. It is an artifact of arithmetic. If a school kicks out students with low test scores, it will have high test scores among the surviving students. If a school culls the students not on track to graduate it will have a high graduation rate among the surviving students.

 

Let’s move on to the big lie. Does this school, in fact, show that the reform strategy of closing large schools and replacing them with other smaller schools works? The full range of data show that it does not, as a number of facts pop-out.

 

There are 4 high schools co-located in the building that used to house Brandeis High School. One school, Innovation Diploma Plus (a “second-chance” school), had a 50.8% graduation rate last year. Another school, the Global Learning Collaborative, had a 52.7% graduation rate last year. Yet another school, the Urban Assembly School for Green Careers had a 39.8% graduation rate last year. We have already examined the claimed 97% graduation rate of the Frank McCourt High School, which also happens to screen its students before admissions.

 

The schools with the lower graduation rates retain almost all of their students. Unlike the school that boasts of its 97% graduation rate, the other three schools stay committed to all their students. Why do reformers refuse to evaluate schools based on their sticking with all their students? We know the answer. It is because charter schools and “miracle” schools will then be publicly exposed as largely frauds. So the metrics used to evaluate schools are deliberately constructed in ways that do not capture cohort retention in order to keep the myth alive. And the media agrees to overlook the tremendously high attrition rates at charter schools and other so-called “miracle” schools.

 

It may come as little surprise that the school with the lowest graduation rate has over 22% more English Language Learners, over 14% more students entering high school already overage, and over 40% more Black/Hispanic students than the school with the highest graduation rate.[i] This sheds some light on another reformer strategy. They like to tout free-market principles as they destroy community schools and create choice systems where students end up sorted into schools based on demographic characteristics and prior academic performance. This is not a solution. It does not improve education for all students. All it does is stick students in different containers, isolated from one another, thereby perpetuating a system of haves and have-nots. It is shocking that the reformers, who proclaim education the civil rights issue of our time, would support such an inequitable approach.

 

The total enrollment of all the high schools in the Brandeis High School building is 1,350 students. The shuttered school, Brandeis High School, had over 2,000 students. Where did the missing 600 students go? We know the answer. The missing 600 students were the more challenging students and students who did not get accepted to one of the small choice-in high schools. These students were deliberately sent to a specific group of, usually large, high schools that were then labeled “failures” too and shuttered. The Gates Foundation, an organization that has yet to meet a free-market education reform strategy it doesn’t like, has admitted that the national small school initiative was largely a failure. Despite this, MDRC, a research group in New York City, continues to publish Gates Foundation funded reports claiming that the small schools in New York City work.[ii]

 

It is now clear what New York City was doing during the Bloomberg era. Given the humungous size of the district they were able to play a shell game with students by passing the buck. Instead of figuring out how to reach the most challenging students and helping them succeed, the students were passed from school to school. This inevitably led to a domino effect of school closures. A shell game like this can be played in a district with almost 500 high schools, over two and half times as many as the next largest school district. Since there is a very long chain of dominos the “bad apple” students can be isolated into a specific group of schools making the remaining schools, which don’t accept those students, look good. But, as smaller districts have found out, it is not a workable long-term strategy when there is not an endless supply of schools to be used as sacrificial lambs.

 

Sooner or later the lies about numbers that reformers tell will catch up to them. Educators need to continue to advocate for approaches that are equitable and genuinely seek to improve the educational experience of all students. This includes developing curricula personalized for different students and improving wraparound services that extend beyond school walls. Ultimately, when the accounting fraud that is behind so many education reform initiatives collapses upon itself

 

[i] Frank McCourt has 55% Black/Hispanic students, 1% ELL students, 20.4% IEP students and .7% overage students. Global Learning has 90.3% Black/Hispanic students, 14.7% ELL students, 23% IEP students and 8.5% overage students. Green Careers has 95.6% Black/Hispanic students, 23.6% ELL students, 23.8 IEP students, and 15.15% overage students.

 

[ii] It is worth noting that the combined graduation rate of the 4 schools in the Brandies Building is 58.5% which is lower than the city-wide average of 72%.

Common Core standards are usually described in the mainstream media in idealistic terms, using the positive and affirmative messages to sell the idea to the public. Doesn’t everyone want “high standards?” Doesn’t everyone want every single student to be “college and career ready?” Doesn’t everyone want students to be “globally competitive”? Of course. These claims, though untested and unproven, sound poll-tested.

Can standards be both “common” and “high”? If they are truly high and rigorous, won’t a sizable proportion of students fail? Can a single set of standards make everyone college and career ready? How do we know? What does it mean to be “globally competitive” with nations where educated people are paid a fraction of our own minimum wage?

Another way to view the Common Core standards is to see them as part of an integrated system of standards, tests, and teacher ratings that generate data. This data can be used to award bonuses, fire teachers, close schools, and identify students for remediation or college admission. The underlying assumption behind CCSS is that all children, if exposed to common standards, will learn at the same pace.

This post challenges the data-driven approach to school reform. “Data,” it says, “is the fool’s gold of the Common Core.”

He writes:

“Teachers should strive to meet the individual needs of their students, not the “needs” of standards or tests. There should be high academic expectations for all students, but to expect everyone, regardless of ability/disability, to meet those standards simultaneously and in the same way is foolish and inherently unfair.

“Standardized tests are toxic for the Common Core and they are the primary reason for the botched implementation efforts around the country. These tests do not generate comprehensive or reliable data regarding constructivist learning that is called for in the Learning Standards….

“The Common Core testing regime is more about satisfying data-driven enthusiasts’ ‘thirst” for more data, than it is about cultivating students’ thirst for knowledge.

“We are witnessing an unprecedented data collection “gold rush”, while the validity and reliability of this “fool’s gold” is of little concern to those who are mining it.

“The “college and career readiness” mandate or mission of the Common Core is misguided and not in the best interest of all our students. There are many “paths” to trade and vocational careers, and they don’t all go through college.

“Since the Common Core Standards were designed to serve and support the college and career readiness mandate, they are seriously flawed and deficient.

“A more inclusive and appropriate mandate such as readiness for “adulthood and employment” would better serve the academic, social, and emotional needs of all our students. Rather than simply “correcting” the inadequate Common Core standards, they should be reconstructed and redesigned from the ground up.”

Mr. Anonymous, an education policy analyst who is working towards his doctorate, wrote the following cautionary story about the use and misuse of statistics for political purposes. He requires anonymity for the usual reasons, mostly fear of retaliation for speaking up.

He writes:

The Common Core and Departments of Education: Lies, Darn Lies, Statistics and Education Statistics

Numbers have taken center stage in the discussion of education policy in the United States. Test score metrics have become a particularly critical set of numbers. They are seen as objective measuring devices, comparable across years, that provide a reliable evaluation of how students, teachers, schools, districts, and the United States as whole are doing. But are they really objective?
The push for implementation of Common Core exams has caught the attention of the public. In New York State, as in many other states across the nation, questions have been raised about the motivations of those pushing for the roll-out of these exams and their use in high-stakes evaluations. As we will see below such concerns are definitely legitimate given the history of the New York State Department of Education and the Board of Regents in setting cut-scores and changing exams in ways that serve political and other ends.

Let’s start with Biology, a standard course that almost every high school freshman takes. Remember dissecting that frog? In 2001 the New York State Department of Education changed the Biology Regents to a re-named “Living Environment.” A rather remarkable aspect of the change was the dramatic lowering of the passing score. In the Biology exam a student needed at least 59 points (out of a total of 85 possible points) to earn a passing grade of 65. On the new Living Environment Regents students need only 40 points (out of a total of 85 possible points) to earn a passing grade of 65. In some years (e.g. 2004) a student needed only 38 out of 85 points to earn a passing grade of 65.

The story repeats itself in mathematics. Until 2002 the New York State Department of Education required students to take a “Sequential Mathematics I” exam. That test had a total point value of 100 points. The conversion was simple enough, each point was equal to one point and a student needed 65 points to pass. Then, in 2002, the math exam was switched to a “Mathematics A” exam.

On this test students needed to score 35 out of a possible 84 points to earn a 65 and pass. Earning 42% of the possible points led to a 65. Then, in 2008, the math exam was switched again, this time to an “Integrated Algebra” exam. On this test students needed to earn 30 out of a possible 87 points to earn a 65 and pass. Earning 34% of the possible points now led to a 65.
The United States and Global History exams underwent similar changes at the turn of the millennium. Before the changes students were required to write 3 essays accounting for 45% of their final score. After the changes students were required to write only two essays accounting for only 35% of their final score. On one of the essays students are provided with extensive information they can use in their writing.

A couple of years later the exact same process occurred with the English Regents. In 2011 the New York State Department of Education changed the exam from a two part six hour test with two essays to a single part three hour test with only one essay. Again the cut scores were dramatically lowered. The scales on these two exams are very different making comparison difficult. One way to measure the change is to look at the grade a student would receive if s/he got exactly half the multiple choice questions correct and earned exactly half of the possible points on the essay(s). On the old English exam that student would have received a grade of 43. On the new English exam a grade of 50.

A year ago the New York State Department of Education changed things yet again. But this time they did not change the exam. They just changed the cut scores. From 2011 until 2013 out of 286 possible point combinations on the exam an average of 74 resulted in a passing grade. Then, in June of 2013, the number of point combinations leading to a passing grade was dramatically lowered by 23%. Since then an average of 63 point combinations out of 286 leads to a passing grade.

It is disturbing that this change occurred at the very moment when the test results would first be used to evaluate teachers. The research base shows that such value-added metrics are unreliable. For example a RAND report concluded “the research base is currently insufficient for us to recommend the use of VAM for high-stakes decisions.” A report out of Brown University concluded “the promise that value-added systems can provide such a precise, meaningful, and comprehensive picture is not supported by the data.” Nonetheless New York State passed laws requiring school districts to use test scores in teacher evaluations. Why, at the same time, did the Department of Education quietly change the cut scores on the English Regents? Is it an attempt to ensure that more teachers are rated ineffective? This would allow certain interest groups to declare the law a success and claim that “bad teachers” are now being identified and should be fired. Is it an attempt to create evidence that there is an epidemic of failing students in New York State? This would allow certain interest groups to proclaim that the crisis can only be solved if the new Common Core Standards are implemented without delay.

Advocates of the Common Core are either ignorant of or deliberately ignore this history. A decade ago New York State Department of Education decided that the high school graduation rate was too low. They therefore changed exams and cut scores to make them easier. The graduation rate went up. Now it seems that some powerful interests have decided that it is too easy to graduate. So they want the exams made harder and the passing cut scores raised. It is evident from the history reviewed above that playing with cut scores is not the way to improve education. After all that just leaves us in the very place we are in today. Yet we seem to be condemned to repeat this cycle all over again. We seem to be enamored of easy solutions. Make exams harder (or easier). Raise cut scores (or lower them). What we do not seem to be willing to do as a nation is roll up our sleeves and do the really, really hard work of ensuring that every student receives a quality education.

Jason France (aka blogger Crazy Crawfish) writes here about the warping and destruction of data held by the Louisiana Department of Education.

He writes:

“There is a data crisis at LDOE. Almost all of the data collection systems are failing. The data, statistics and reports being generated are garbage. Data is being ferried back and forth between the department and school districts using Excel worksheets and through e-mail correspondence. This leaves many students at high risk to data theft and privacy violations. Because the systems impacted are numerous and core to much of the reporting and analysis performed by the Department, it is impossible for LDOE to claim they are reporting accurate or reliable numbers for dropouts, graduates, TOPS scholarship awards, school performance scores, test scores, student counts and breakdowns for MFP funding, program counts. . . the list goes on and on. The situation is really serious and probably just about hopeless at this point.

“I will explain how this situation developed and give specific examples of systems, impacted and correspondence I’ve received from school districts trying to work with the department.

“This crisis was created intentionally by John White and his second in command that he brought with him from New York, Kunjan Narechania. White did not really care what the data said, because he had already determined the outcome for many of his programs. (I don’t think he was also not planning to be here longer than 2 years when all the cut-backs and destruction he’d wrought really started to impact daily operations.) White undertook a slash and burn campaign on the department’s data and analysis folks and immediately implemented policies that guaranteed data would deteriorate immediately. White abandoned a 4 million dollar warehouse named LEDRS we were just finishing. . . as he arrived on the scene, but not before using it to transmit almost all of the data contained in the Warehouse to CREDO to produce reform friendly propaganda masquerading as true data analysis.”

Read on for a remarkable story.

Yesterday, the New York Times published an editorial vigorously agreeing with the Obama administration’s plan to give ratings to colleges and universities and agreeing with Education Trust that federal aid to colleges should be tied to those ratings. EdTrust was and remains one of the strongest supporters of NO Child Left Behind, having helped to write that abominable law.

On principle, I oppose the ratings game and believe it will turn into NCLB for higher education, with incentives that undermine the mission of the institutions as they get caught up in the numbers game. How will colleges measure the “value-added” of courses in philosophy, ancient history, art, and music?

The Times apparently doesn’t read its own stories. It doesn’t recognize that students are discouraged not by a lack of information but by the crushing debt they incur. Why don’t we have low-cost or free public colleges? The Times has reported in the past about how states have shifted costs from the public to students. Why is this not a more pressing need than data?

When the Times says that the U.S. has among the lowest college graduation rates in the developed world, it should have mentioned that one nation with a much lower rate is Germany, the dominant economy in Europe. What does Germany know that we don’t know?

If the Times thinks that getting a higher college graduation rate matters, why not propose ways to reduce the cost to students? The greatest barrier to college access and completion is affordability, not lack of data.

Stephanie Simon writes in politico.com about how parents organized, lobbied, agitated, and brought down inBloom.

Simon writes:

“You’ve heard of Big Oil and Big Tobacco. Now get ready for Big Parent.

“Moms and dads from across the political spectrum have mobilized into an unexpected political force in recent months to fight the data mining of their children. In a frenzy of activity, they’ve catapulted student privacy — an issue that was barely on anyone’s radar last spring — to prominence in statehouses from New York to Florida to Wyoming.”

Most shocking of all is that the Obama administration is prepared to spend $1 billion (half from the federal government, half from the states) to track the movements of every child:

“Now, parents are rallying against another perceived threat: huge state databases being built to track children for more than two decades, from as early as infancy through the start of their careers.

Promoted by the Obama administration, the databases are being built in nearly every state at a total cost of well over $1 billion. They are intended to store intimate details on tens of millions of children and young adults — identified by name, birth date, address and even, in some cases, Social Security number — to help officials pinpoint the education system’s strengths and weaknesses and craft public policy accordingly.

“The Education Department lists hundreds of questions that it urges states to answer about each child in the public school system: Did she make friends easily as a toddler? Was he disciplined for fighting as a teen? Did he take geometry? Does she suffer from mental illness? Did he go to college? Did he graduate? How much does he earn?

“Every parent I’ve talked to has been horrified,” said Leonie Haimson, a New York mother who is organizing a national Parent Coalition for Student Privacy. “We just don’t want our kids tracked from cradle to grave.”

Why does the Education Department want so much information about every child? What is the rationale for assembling Big Data about our children? Does Congress know about this? Is there any other government in the world that is data mining its children?

Will parents mobilize to stop the federal government from mining their children’s personal data?

Bob Shepherd writes on the absurd demands now placed on teachers and principals by politicians, who expect to see higher test scores every year. Step back and you realize that the politicians, the policy wonks, the economists, and the ideologues are ruining education, not improving it. They are doing their best to demoralize professional educators. What are they thinking? Are they thinking? Or is it just their love of disruption, let loose on children, families, communities, and educators?

Bon Shepherd writes:

OK, you are sitting in your year-end evaluation session, and you’ve heard from every other teacher in your school that his or her scores were a full level lower this year than last, and so you know that the central office has leaned on the principal to give fewer exemplary ratings even though your school actually doesn’t have a problem with its test scores and people are doing what they did last year but a bit better, of course, because one grows each year as a teacher–one refines what one did before, and one never stops learning.

But you know that this ritual doesn’t have anything, really, to do with improvement. It has to do with everyone, all along the line, covering his or her tushy and playing the game and doing exactly what he or she is told. And, at any rate, everyone knows that the tests are not particularly valid and that’s not really the issue at your school because, the test scores are pretty good because this is a suburban school with affluent parents, and the kids always, year after year, do quite well.

So whether the kids are learning isn’t really the issue. The issue is that by some sort of magic formula, each cohort of kids is supposed to perform better than the last–significantly better–on the tests, though they come into your classes in exactly the same shape they’ve always come into them in because, you know, they are kids and they are just learning and teaching ISN’T magic. It’s a lot of hard work. It’s magical, sometimes, of course, but its’ not magic. There’s no magic formula.

So, the stuff you’ve been told to do in your “trainings” (“Bark. Roll over. Sit. Good Boy”) is pretty transparently teaching-to-the-test because that’s the only way the insane demand that each cohort will be magically superior to the last as measured by these tests can be met, but you feel in your heart of hearts that doing that would be JUST WRONG–it would short-change your students to start teaching InstaWriting-for-the-Test, Grade 5, instead of, say, teaching writing. And despite all the demeaning crap you are subjected to, you still give a damn.

And you sit there and you actually feel sorry for this principal because she, too, is squirming like a fly in treacle in the muck that is Education Deform, and she knows she has fantastic teachers who knock it out of the park year after year, but her life has become a living hell of accountability reports and data chats to the point that she doesn’t have time for anything else anymore (she has said this many times), and now she has to sit there and tell her amazing veteran teachers who have worked so hard all these years and who care so much and give so much and are so learned and caring that they are just satisfactory, and she feels like hell doing this and is wondering when she can retire.

And the fact that you BOTH know this hangs there in the room–the big, ugly, unspoken thing. And the politicians and the plutocrats and the policy wonks at the Thomas B. Fordham Institute and the Secretary of the Department for the Standardization of US Education, formerly the USDE, and the Vichy education guru collaborators with these people barrel ahead, like so many drunks in a car plowing through a crowd of pedestrians.

Follow

Get every new post delivered to your Inbox.

Join 108,248 other followers