Archives for category: Data and Data Mining

Danielle Dreilinger of the Times-Picayune reports that Louisiana’s graduation rate is deeply flawed by missing data in several districts.

When students transfer out, are they leaving for private school, home school, another school, or another state?

“Education officials audited 2012-13 transfer records of 34 of the state’s 69 systems and found one third of these exits could not be properly documented.”

“Much is at stake with the record-keeping, for students must be considered dropouts if their transfers are not properly documented. That depresses the school’s graduation rate. But if the transfer papers are in order, the student is not counted in their high school’s graduation rate.

“The graduation rate counts for 25 percent of the school performance score. That score determines whether conventional schools may be taken over by the state and whether charter schools may stay open.

“The worst results in the audit were found in Jefferson and East Baton Rouge parishes, and in the New Orleans schools run directly by the Recovery School District. Only 27 percent of East Baton Rouge transfer records — and none of the Recovery-New Orleans records — had proper documentation. In Jefferson, the verification rate was 31 percent.”

Veteran educator Mike Deshotels believes that school officials might be cooking the books. The state, of course, vigorously denies that claim.

We should have learned from Bernie Madoff and Ken Lay of Enron that data are highly malleable.

Which is the most powerful player behind the scenes in corporate reform?

This article says, without doubt, McKinsey.

Where did David Coleman, architect of the Common Core standards, get his start: McKinsey.

Which firm pushes the narrative of a “crisis in education”: McKinsey.

Which firm believes that Big Data will solve all problems? McKinsey.

Look behind the screen, behind the curtain: McKinsey.

A teacher in Texas wrote this comment, which depicts (to me) a system where data matters more than teachers or learning or children, either the system is on autopilot or is run by people who confuse numbers with learning.

“They recruited from NC and from Spain (for bilingual teachers) this year because they did expect vacancies. I think it’s important to mention that all are not based on EVAAS because not everyone has those standardized scores. They are also based on Stanford testing in 1st and 2nd grade and for classes like PE, a district made assessment. I teach Kinder and am still waiting to find out what growth they calculated for my scores last year (and yes, they were bubble-in multiple choice tests). No one could explain to me how it was going to work, what percentage growth was required to be considered effective and how that was going to be calculated– so I’m very anxious about it. I was rated highly effective in the professional and instructional areas but who knows. We are supposed to use 2 different assessments for more validity but that doesn’t happen-they end up using the reading and math versions of the same test given the same week. I did wonder how many vacancies they had to start the new school year yesterday?”

Joel Westheimer is a professor at the University of Ottawa in Canada, where he serves as the University Research Chair in Democracy in Education. He wrote this tribute to Mr. Keating, the fictional teacher in “The Dead Poets’ Society” before Robin Williams’ death.

 

 

He wrote:

 

 

 

 

In a popular scene from the 1989 movie, Dead Poets Society, the eccentric Mr. Keating (played by Robin Williams), asks one of his students to read aloud from the preface of a high school poetry textbook:

 

To fully understand poetry, we must first be fluent with its meter, rhyme and figures of speech, then ask two questions: 1) How artfully has the objective of the poem been rendered and 2) How important is that objective? Question 1 rates the poem’s perfection; question 2 rates its importance. And once these questions have been answered, determining the poem’s greatness becomes a relatively simple matter. If the poem’s score for perfection is plotted on the horizontal of a graph and its importance is plotted on the vertical, then calculating the total area of the poem yields the measure of its greatness.
The fictional author of the text, Dr. J. Evans Pritchard, PhD, continues with an example: “A sonnet by Byron might score high on the vertical but only average on the horizontal. A Shakespearean sonnet, on the other hand, would score high both horizontally and vertically, yielding a massive total area, thereby revealing the poem to be truly great.” Pritchard concludes by asking students to practice this rating method (using the provided rubric) because “[a]s your ability to evaluate poems in this matter grows, so will your enjoyment and understanding of poetry.”

 

Although both the textbook and its author are fictional, the satire is worrisomely apt. In fact, the fictional passage was based closely on a real text found in a popular 1950s poetry textbook currently in its twelfth edition and still used by high school students across the country: Laurence Perrine’s Sound and Sense: An Introduction to Poetry. In other words, the demand for standardized measures of quality and success in education has not abated but increased

 

The relatively uncritical and universal acceptance among school reformers of the importance of so-called standards, rubrics, and uniform assessment tools for teaching and learning is at once predictable and misguided. It is predictable because the idea that we should clearly articulate educational goals and then devise methods for determining whether those goals are met is irresistibly tidy. After all, how can teachers pursue high quality lessons if they do not know what they are trying to teach and whether students are learning? Uncritical acceptance of even such a common-sense seeming idea, however, is misguided for the following reason: education is first and foremost about human relationship and interaction, and as anyone who tried to create a standardized test for family fealty or for love or for trust would discover, any effort to quantify complex human interactions quickly devolves into a fool’s errand.

 

This does not mean that there is no place for evaluation in education, or for standards, rubrics, and common curriculum frameworks. A new book, Rubric Nation, coming out this Fall edited by Joseph Flynn and Michelle Tenem-Zemach takes a critical stance at the same time many of the contributing authors make the need for thoughtful measures and learning frameworks clear.[i] Moreover, I have rarely met a teacher who did not have standards; most have their own forms of rubrics or evaluative frameworks as well. But “No Child Left Behind” and “Race to the Top” legislation and related reforms that call for ever-more standardized rubrics and frameworks have severely restricted teachers’ abilities to act in a professional capacity and exercise professional judgment on behalf of their students.

 

Finnish educator Pasi Sahlberg calls the kind of school reform that elevates the pursuit of rubrics and standardization above all other educational considerations GERM (for Global Education Reform Movement). He describes GERM as follows:

 

It is like an epidemic that spreads and infects education systems through a virus. It travels with pundits, media and politicians. Education systems borrow policies from others and get infected. As a consequence, schools get ill, teachers don’t feel well, and kids learn less.[ii]
Not only do kids learn less. What they learn also tends to follow prescriptive formulas that match the standardized tests. In the process, more complex and difficult-to-measure learning outcomes get left behind. These include creativity and emotional and social development as well as the kinds of thinking skills associated with robust civic engagement. As a result, teachers’ ability to teach critical thinking and students’ ability to think and act critically is diminished.

 

Almost every school mission statement these days boasts broad goals related to critical thinking, global citizenship, environmental stewardship, and moral character. Yet beneath the rhetoric, increasingly narrow curriculum goals, accountability measures, standardized testing and an obsession with rubrics have reduced too many classroom lessons to the cold, stark pursuit of information and skills without context and without social meaning – what the late education philosopher Maxine Greene called mean and repellent facts. It is not that facts are bad or that they should be ignored. But democratic societies require more than citizens who are fact-full. They require citizens who can think and act in ethically thoughtful ways. Schools need the kinds of classroom practices that teach students to recognize ambiguity and conflict in “factual” content and to see human conditions and aspirations as complex and contested.

 

As our cultural obsession with standardization, rubrics, and accountability measures in only two subject areas (math and literacy) increasingly dominates school reform, the most common complaint I now hear from both teachers and administrators is this: I have been stripped of my professional judgment, creativity, and freedom to make decisions in the best interests of my students. When education reforms turn away from an emphasis on supporting positive conditions of practice and move towards technocratic strategies for “compliance,” the profession suffers and so do students. Many teachers would echo the sentiments of Gloria, a teacher in a recent study I conducted of the 10th grade civics curriculum in Ontario. She told us this:

 

In my 22 years of teaching, never have I experienced a climate that has turned all educational problems into problems of measurement until now. Poor citizenship skills? Raise their math and literacy scores. Poor participation? Doesn’t matter. Poverty? Inequality? The solution is always always to give the students more tests. These days pedagogically, I feel like I can’t breathe.
But education goals, particularly in democratic societies, have always been about more than narrow measures of success, and teachers have often been called upon and appreciated for instilling in their students a sense of purpose, meaning, community, compassion, integrity, imagination, and commitment. Every teacher accomplishes these more artful and ambiguous tasks in different ways.

 

Much as Darwin’s theory of natural selection depends on genetic variation, any theory of teaching in a democratic society depends on a multiplicity of ideas, perspectives, and approaches to exploring and seeking solutions to complex issues of widespread concern. Parents, administrators, and politicians alike all must acknowledge that educators in a democratic society have a responsibility to create learning environments that teach students a broad variety of lessons – including but not limited to the kinds of learning goals easily captured by standardized assessments.

 

Talented teachers need the freedom and professional autonomy to work the magic of their art in a myriad of different ways that defy standardization and regimentation of practice. Talented teachers need manageable class sizes in which they can provide the right conditions for that magic to take root. And talented teachers need policymakers who have the courage to marshal the resources necessary to create the best possible conditions of practice and then let teachers do their jobs free of interference and corrosive mistrust.

 

Nothing about the kinds of standards that school reformers are pursuing with such certainty is black and white. That’s why scholars of education must work together to create a space for dialogue around the tensions inherent in the teaching profession between autonomy and committee-rule, between spontaneity and uniformity. Far from allowing the poetry of teaching and learning to be reduced to facile measurements, educators must demand a fuller framing of assessment and educational progress.

 

You may recall in Dead Poets Society that after allowing his students to listen attentively to the detailed instructions on measuring the quality of poetry (even drawing a graph on the blackboard to show just how to execute the formula for evaluation), Keating proceeds to demand that students rip out that entire chapter from the text. “Be gone J. Evans Pritchard, PhD!” he exclaims to the sound of students tearing out the offending pages. He was asking them, of course, to revel in the radical possibility of unquantifiable teaching and learning. In honor of Mr. Williams’ irreverent humor and his complex portrayal of Mr. Keating, I hope every teacher enters the new school year with just such an attitude.

 

 

[i] J. Flynn & M. Tenam-Zemach (Eds.), Rubric Nation: A reader on the utility and impact of rubrics in education. Charlotte, North Carolina: Information Age Publishing.

 

[ii] Sahlberg, P. (2012, June 29). How GERM is infecting schools around the world. The Washington Post. [The Answer Sheet web log by Valerie Strauss]. Retrieved http://www.washingtonpost.com/blogs/answer-sheet/post/how-germ-is-infecting-schools-around-the-world/2012/06/29/gJQAVELZAW_blog.html.

Andy Hargreaves of Boston College asks an important question: What is the purpose of benchmarking? We collect data, we measure, we test, we set goals, but why? Will it improve performance if we know that someone else does it better? Do they have the same challenges, the same resources? Is there more to education than raising tests ores and do higher test scores necessarily mean better education?

Andy begins with two stories about benchmarking, one positive, one negative. One improved public health, one made it easier to conduct war.

Right now, under pressure from No Child Left Behind and Race to the Top, everything is measured. Why? To fire teachers and principals? To close schools? To hand public property to entrepreneurs? Who benefits? What do we do with the losers? Throw them away? Plenty of children were left behind, and many will not make it to “the top.”

Andy writes:

“Is the purpose of our educational benchmarking to further the public good, to raise the standards of education for all, to elevate the poorest and most disadvantaged students to the greatest heights of accomplishment? And once we have done our calculations and made our maps, what pathways will be opened up, and what people and resources will be pulled along them in this worthy quest for equity and excellence? The White House announced earlier this summer that it would address educational inequities by collecting data to help pinpoint where they existed, but there seemed to be no plan to bring up the people and resources to correct them.

“Is there a second purpose of educational benchmarking then? Is it to delineate the weak from the strong, inciting nation to compete against nation, Americans against Asians, and school against school. After we have pinpointed schools that are failing, does this just make it easier for invading opportunists to set up charter schools in their place, or to market online alternatives, tutoring services and the like?

“As in surveying, benchmarking in education should be about discovering where we stand and learning about who we are and what we do by observing those around us. It should be about improving public education, just as the sewer maps for my hometown contributed to public sanitation. Benchmarking should not be about fomenting panics about performance in relation to overseas competitors. And it should not be about dividing schools, families and communities from each other to create easy pickings for the educational market.

“Whenever we are engaged in the data-driven detail of educational benchmarking, these are the greater questions we should be asking. Of what map or whose map are we the servants?”

On Anthony Cody’s new independent blog site, “Living in Dialogue,” Chicago teacher Michelle Gunderson offers her views on the ethical use of student data. 

 

In her many years as an elementary school teacher, she has seen standardized tests evolve from a sorting instrument to a means of punishing children to an excuse for privatizing public schools.

 

She will not be complicit in any of these uses of student test scores. She would abolish the standardized tests if she could, but that is not within her power.

 

So she pledges, first, that they will always be on of multiple measures; that she will remain strict confidentiality about student test scores and never publish them on a data wall or release them to the public; and that she will communicate with families about the frequency and amount of time spent on testing.

 

Tests, like all tools, may be used wisely or wrongly. Tests should be used to help children and teachers, not to punish or label them or close their school.

If you have not read Rachel Aviv’s “Wrong Answer” in The New Yorker, drop everything and read it now.

Aviv tells the story of the Atlanta cheating scandal through the ideas of one man, one teacher, who cared deeply about his student. Step by step, he got sucked into the data-driven obsession with test scores, thinking that if he raised the children’s test scores, it was a victimless crime. He knew that his students had needs that were even greater than their test scores, but the law’s absurd requirement that scores had to go up year after year drew him into a widespread conspiracy to falsify test scores.

One day will we look back on the Atlanta cheating scandal as the wake up call that made us think about how successive administrations and members of Congress have given their approval to laws and goals that hurt children and warped education? Or will we continue on the present path of destruction?

Marion Brady is a retired teacher and administrator and prolific author.

He writes:

“In a commentary in the July 21, 2014 issue of Time magazine, columnist Joe Klein takes aim at one of the usual targets of today’s education reformers—unions. In a dig at New York City mayor de Blasio, he says, “A mayor who actually cared about education would be seeking longer school days, longer school years, more charter schools…and the elimination of tenure and seniority rules…”

“Like just about every other mainstream media pundit, Klein thinks he knows enough about educating to diagnose its ills and prescribe a cure. That he’ll be taken seriously testifies to the power of what’s become the conventional wisdom, that if America’s schools aren’t performing as they should it’s because teachers aren’t getting the job done.

“What’s the teacher’s job? Raising standardized tests scores.

“What’s the key to high test scores? Rigor.

“What does rigor look like? No-excuses teachers doing their thing for as long as it takes to get the job done.

“What’s “their thing”? Teaching to demanding standards—the Common Core State Standards.

“The market-force-education-reform juggernaut set in motion by business leaders and politicians about a quarter-century ago is simple and easily summarized. (1) Adopt tough performance standards for school subjects. (2) Use high-stakes tests to measure performance. (3) Reward high-scorers; punish low scorers.

“Which, when you think about it, is off the mark. School subjects are just tools—means to an end. We don’t tell surgeons which scalpels and clamps to use; what we want to know is their kill/cure rate. We don’t check the toolbox of the plumber we’ve called to see if he (or she) brought a basin wrench and propane torch; we want to know that when the job’s done the stuff goes down when we flush. We don’t kick the tires of the airliner we’re about to board; we trust the judgment of the people on the flight deck.

“School subjects are tools. Kids show up for kindergarten enormously curious and creative. What we need to know is how well schooling is enhancing that curiosity and creativity. Kids learn an incredible amount on their own long before they walk through school doors. What we need to know is how much improvement there’s been in self-directed learning. Kids appear to begin life with an innate sense of what’s right and fair. What we need to know is how successfully that sense is being nurtured.

“We’re on a wrong track. Standards? Of course! But not standards for school subjects. What’s needed are standards for the qualities of mind, emotion, character, and spirit the young must be helped to develop if they’re to cope with the world they’re inheriting.
The Common Core Standards, says the CCSS website, “provide clear signposts along the way to the goal of college and career readiness.” Just stick to the CCSS script to be prepared for college and career.

“College? Years ago, the Association of American Colleges’s Project on Redefining the Meaning and Purpose of Baccalaureate Degrees said, “We do not believe that the road to a coherent education can be constructed from a set of required subjects or academic disciplines.” I’ve seen no evidence that the thoughtful among them have changed their minds.

“Careers? We have no idea how the interactions of globalization, automation, climate change, clashing societal
worldviews, and trends not yet evident will effect careers. The only thing that can be said with certainty is that nobody knows what careers are going to be available when today’s elementary school kids are looking for work.

“Back in the 70s, in his book Reflections on the Human Condition, Eric Hoffer, philosopher, writer, and longshoreman, wrote something that the Common Core Standards don’t adequately reflect: “In a time of drastic change it is the learners who inherit the future. The learned usually find themselves equipped to live in a world that no longer exists.”

“Standards? Sure. But not standards for solving quadratic equations, or for recalling the chemical formulas for salt, sand, baking soda, and chalk, or for interpreting Dr. King’s Letter from a Birmingham Jail as some self-appointed “expert” thinks it should be interpreted.
And not standards that make it easy to create machine-scored tests that perpetuate the destructive myth that quality can be quantified and turned into data to drive education reform.

“Standards—proper standards—could work wonders. Consider, for example, the effect just one standard could have on teachers, on teaching materials, on kids, on the citizenry, on America:

“Schools will be held accountable for sending learners on their way with a deep-seated love of learning and a willingness and ability to follow where that love leads.”

I did not write the following post. It was written by a high-level official at the New York City Department of Education who–for obvious reasons–requires anonymity. The story he tells is instructive. It is about how “reformers” claim victory by manipulating statistics. This is not an accusation directed at the de Blasio administration, but at their predecessors who regularly boasted that the new small high schools got better graduation rates that the large schools they replaced. The Gates Foundation bought this lie and has lauded its “success” in New York City.

 

 

 

 

Reformers Caught Lying. Again. This Time About Graduation Stats.

 

High school graduations are upon us. This is the time of year when parents, students, families, educators and communities celebrate the accomplishments of our high school seniors. It is a time to honor the work the graduates have done and to collectively share their hopes and dreams for the future.

 

At the same time, certain players in the world of education attempt to co-opt this time of year to propagandize for their favored reform policies. The latest example of this is a story about Frank McCourt High School, a small high school in New York City that “will send 97 percent of its first graduating class to college.” The story goes on to note that this school, along with 3 others, replaced a larger high school “which suffered from dismal academic and attendance records.” The reader is asked to believe one little and one big lie.

 

Let’s first take a look at the little lie. Does this school, in fact, have a 97% graduation/college going rate? The truth is that, on any reasonable calculation, it does not. According to the New York State data the cohort started with 100 freshmen. By sophomore year only 88 students remained. By junior year only 80. And by senior year only 69. Of these 69 survivors 67 are graduating. Seems more like a twisted version of the Hunger Games than a school for all students. One wonders: Where did the other 33 students go? Why does the media publicize such meaningless numbers without giving the full story? By now this trick should be well-known. A school that removes large numbers of students from its cohort should not be celebrated for its test scores or graduation rate. It is an artifact of arithmetic. If a school kicks out students with low test scores, it will have high test scores among the surviving students. If a school culls the students not on track to graduate it will have a high graduation rate among the surviving students.

 

Let’s move on to the big lie. Does this school, in fact, show that the reform strategy of closing large schools and replacing them with other smaller schools works? The full range of data show that it does not, as a number of facts pop-out.

 

There are 4 high schools co-located in the building that used to house Brandeis High School. One school, Innovation Diploma Plus (a “second-chance” school), had a 50.8% graduation rate last year. Another school, the Global Learning Collaborative, had a 52.7% graduation rate last year. Yet another school, the Urban Assembly School for Green Careers had a 39.8% graduation rate last year. We have already examined the claimed 97% graduation rate of the Frank McCourt High School, which also happens to screen its students before admissions.

 

The schools with the lower graduation rates retain almost all of their students. Unlike the school that boasts of its 97% graduation rate, the other three schools stay committed to all their students. Why do reformers refuse to evaluate schools based on their sticking with all their students? We know the answer. It is because charter schools and “miracle” schools will then be publicly exposed as largely frauds. So the metrics used to evaluate schools are deliberately constructed in ways that do not capture cohort retention in order to keep the myth alive. And the media agrees to overlook the tremendously high attrition rates at charter schools and other so-called “miracle” schools.

 

It may come as little surprise that the school with the lowest graduation rate has over 22% more English Language Learners, over 14% more students entering high school already overage, and over 40% more Black/Hispanic students than the school with the highest graduation rate.[i] This sheds some light on another reformer strategy. They like to tout free-market principles as they destroy community schools and create choice systems where students end up sorted into schools based on demographic characteristics and prior academic performance. This is not a solution. It does not improve education for all students. All it does is stick students in different containers, isolated from one another, thereby perpetuating a system of haves and have-nots. It is shocking that the reformers, who proclaim education the civil rights issue of our time, would support such an inequitable approach.

 

The total enrollment of all the high schools in the Brandeis High School building is 1,350 students. The shuttered school, Brandeis High School, had over 2,000 students. Where did the missing 600 students go? We know the answer. The missing 600 students were the more challenging students and students who did not get accepted to one of the small choice-in high schools. These students were deliberately sent to a specific group of, usually large, high schools that were then labeled “failures” too and shuttered. The Gates Foundation, an organization that has yet to meet a free-market education reform strategy it doesn’t like, has admitted that the national small school initiative was largely a failure. Despite this, MDRC, a research group in New York City, continues to publish Gates Foundation funded reports claiming that the small schools in New York City work.[ii]

 

It is now clear what New York City was doing during the Bloomberg era. Given the humungous size of the district they were able to play a shell game with students by passing the buck. Instead of figuring out how to reach the most challenging students and helping them succeed, the students were passed from school to school. This inevitably led to a domino effect of school closures. A shell game like this can be played in a district with almost 500 high schools, over two and half times as many as the next largest school district. Since there is a very long chain of dominos the “bad apple” students can be isolated into a specific group of schools making the remaining schools, which don’t accept those students, look good. But, as smaller districts have found out, it is not a workable long-term strategy when there is not an endless supply of schools to be used as sacrificial lambs.

 

Sooner or later the lies about numbers that reformers tell will catch up to them. Educators need to continue to advocate for approaches that are equitable and genuinely seek to improve the educational experience of all students. This includes developing curricula personalized for different students and improving wraparound services that extend beyond school walls. Ultimately, when the accounting fraud that is behind so many education reform initiatives collapses upon itself

 

[i] Frank McCourt has 55% Black/Hispanic students, 1% ELL students, 20.4% IEP students and .7% overage students. Global Learning has 90.3% Black/Hispanic students, 14.7% ELL students, 23% IEP students and 8.5% overage students. Green Careers has 95.6% Black/Hispanic students, 23.6% ELL students, 23.8 IEP students, and 15.15% overage students.

 

[ii] It is worth noting that the combined graduation rate of the 4 schools in the Brandies Building is 58.5% which is lower than the city-wide average of 72%.

Common Core standards are usually described in the mainstream media in idealistic terms, using the positive and affirmative messages to sell the idea to the public. Doesn’t everyone want “high standards?” Doesn’t everyone want every single student to be “college and career ready?” Doesn’t everyone want students to be “globally competitive”? Of course. These claims, though untested and unproven, sound poll-tested.

Can standards be both “common” and “high”? If they are truly high and rigorous, won’t a sizable proportion of students fail? Can a single set of standards make everyone college and career ready? How do we know? What does it mean to be “globally competitive” with nations where educated people are paid a fraction of our own minimum wage?

Another way to view the Common Core standards is to see them as part of an integrated system of standards, tests, and teacher ratings that generate data. This data can be used to award bonuses, fire teachers, close schools, and identify students for remediation or college admission. The underlying assumption behind CCSS is that all children, if exposed to common standards, will learn at the same pace.

This post challenges the data-driven approach to school reform. “Data,” it says, “is the fool’s gold of the Common Core.”

He writes:

“Teachers should strive to meet the individual needs of their students, not the “needs” of standards or tests. There should be high academic expectations for all students, but to expect everyone, regardless of ability/disability, to meet those standards simultaneously and in the same way is foolish and inherently unfair.

“Standardized tests are toxic for the Common Core and they are the primary reason for the botched implementation efforts around the country. These tests do not generate comprehensive or reliable data regarding constructivist learning that is called for in the Learning Standards….

“The Common Core testing regime is more about satisfying data-driven enthusiasts’ ‘thirst” for more data, than it is about cultivating students’ thirst for knowledge.

“We are witnessing an unprecedented data collection “gold rush”, while the validity and reliability of this “fool’s gold” is of little concern to those who are mining it.

“The “college and career readiness” mandate or mission of the Common Core is misguided and not in the best interest of all our students. There are many “paths” to trade and vocational careers, and they don’t all go through college.

“Since the Common Core Standards were designed to serve and support the college and career readiness mandate, they are seriously flawed and deficient.

“A more inclusive and appropriate mandate such as readiness for “adulthood and employment” would better serve the academic, social, and emotional needs of all our students. Rather than simply “correcting” the inadequate Common Core standards, they should be reconstructed and redesigned from the ground up.”