Archives for category: Campbell’s Law

 

Peter Greene writes here about the “moonshot” to transform American education, co-sponsored by the conservative Thomas B. Fordham Institute and the allegedly liberal Center for AMERICAN Progress. Peter points out that this collaboration demonstrates that both sides of the DC Establishment endorse corporatedceducarion reform (despite its manifest failure for the past 25 years).

He compares their competition to education’s version of the self-driving car.

He writes:

Do you mean something that’s promoted relentlessly but is still far off in the future? Or do you mean a program that faces major obstacles that tech-cheerleaders just sort of gloss over?

Perhaps you meant a tech-based solution that strips all participants of power and agency and gives it instead to a bunch of programmers? Or did you mean a new tech initiative that promises to make a bunch of people rich?

Or do you mean something that can fail with really catastrophic results?

All their goals are stated as measurable results.

And he notes:

These goals are all about changing numbers; they are an open invitation to apply Goodhart’s or Campbell’s Laws, in which focus on a measurement leads to that measurement being rendered useless. This is about coming up with ways to make better numbers. Yes, one way to improve numbers can be (though not always) to improve the underlying reality those numbers are supposed to represent. But those techniques are hard to scale, expensive and not easy to devise. There are always simpler methods.

If you want a piece of this action, the group is open to submissions of 500 words until the end of the month. But remember– this is not about coming up with a self-driving car. It’s about coming up with a marketing package that makes it look like a self-driving car has been perfected. It’s about doing a good job of using modern CGI to fake your presence on the moon without all the hard work, expense and challenge of actually getting a rocket up there.

No Child Left Behind will be recognized in time as the most colossal failure in federal education policy, whose disastrous effects were amplified by Race to the Top.

Its monomaniacal focus on test scores warped education. RTT just made it worse and left a path of destruction in urban districts.

And the gains were, as a new study reports, modest and diminished over time.

Anyone familiar with Campbell’s Law could have predicted this result. Social scientist Donald T. Campbell wrote:

“The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.”

Campbell also wrote:

“Achievement tests may well be valuable indicators of general school achievement under conditions of normal teaching aimed at general competence. But when test scores become the goal of the teaching process, they both lose their value as indicators of educational status and distort the educational process in undesirable ways. (Similar biases of course surround the use of objective tests in courses or as entrance examinations.)”

Scores on NAEP rose modestly for a few years but went flat in 2015 and again in 2017.

Arne Duncan is traversing the country and TV boasting of his success and asserting that American education is built on lies. He should know. The biggest lie was NCLB. The second biggest lie was Race to the Top. The third biggest lie is ESSA.

The belief that threats and rewards will produce better education is not just a lie. It is stupid.

 

John Merrow has followed the D.C. fiasco closely. He has been all over D.C. since he spent three years celebrating Michelle Rhee’s Leadership, then realized she was a fraud.

Here he is again, citing the D.C. sham, concentrated first on test scores, then graduation rates.

“Friends,

“Campbell’s Law teaches us that, when too much pressure is placed on a single measurement, that measurement inevitably becomes corrupted to the point of being useless. A straightforward analogy is to physical health. An individual who worries only about weight is a strong candidate for anorexia and bulimia. On the other hand, the person who pays attention to muscle and skin tone, flexibility, endurance, a balanced diet, daily exercise, and personal appearance–as well as weight–is NOT a candidate for an eating disorder.

“The same principle applies to education: When a system values (and measures) many aspects of schooling, such as the amount of art and music, the time devoted to recess, student attendance, teacher attendance, teacher turnover, and academic achievement, the school and its students, teachers and staff are likely to be ‘balanced.’ When only test scores or graduation rates matter, bad things are guaranteed to happen.

“Evidence of educational anorexia and bulimia isn’t hard to find. The absence of art, music, science, and recess is one clear sign. Lots of test-prep is another clear indicator. Rallies for ‘higher test scores’ is strong evidence. At home, check on your child’s anxiety level. Stomach aches before the days of standardized testing? Trouble sleeping? It’s all right there in front of us.

“Under Washington DC Schools Chancellor Michelle Rhee (2007-2010), all that counted was test scores, and before long adults began cheating, knowing that their jobs depended on raising scores. Under her successor, Kaya Henderson (2010-2016), raising graduation rates became the Holy Grail, and we now know what transpired: hundreds of seniors–one third of the graduating class–were given diplomas even though they had been skipping school regularly or had otherwise not followed the rules. Her successor as Chancellor, Antwan Wilson, not only failed to monitor and correct that situation; he also broke his own rules and illegally transferred his daughter into a selective high school, bypassing the lottery.

“It’s impossible not to conclude that Washington has been sold a bill of goods by ‘reformers’ like Rhee, Henderson and others. That narrative has been widely accepted and spread by the pundit class and former Secretary of Education Arne Duncan.

“The reaction to the DC fiasco has been revealing. Those on the far right, including current Secretary of Education Betsy DeVos, have expressed outrage. DeVos has called for an FBI investigation and for more ‘school choice.’ That’s also been the call from The Manhattan Institute, which claims “The only thing that’s actually worked in Washington, D.C., has been school choice.” Frankly, these guys and gals will do anything they can to undermine public education.”””

Read on.

 

New York State released the latest graduation rate data. The grad rate went up by half a percentage point, from 79.7% to 80.2%.

http://www.nysed.gov/news/2018/state-education-department-releases-2013-cohort-high-school-graduation-rates

It is good to see more students finishing high school, and their teachers and principals and the students themselves deserve congratulations.

But.

Given the spread of online “credit recovery,” which allows students to make up for a failed course by taking an online course for a few days or a couple of weeks, is the grad rate real or is it margarine?

The recent graduation rate scandal in D.C. should raise red flags everywhere.

Future releases should report on the use of credit recovery courses. The NCAA became so concerned about credit recovery that it disqualified high schools that were allowing students to graduate with fake degrees, in some cases received after answering simple true-false questions. In some cases, with multiple chances to take the multiple-choice exam.

Campbell’s Law is immutable.

For many years, Karin Klein wrote editorials about education for the Los Angeles Times. She took a buyout and now writes freelance on education and other topics. With occasional diversions, the L.A. Times faithfully followed Eli Broad’s lead on education. The billionaire is living proof that being very rich qualifies you as an expert on most everything. He spends lavishly on art and medical research and has anointed himself an education expert. His foundation gives the L.A. Times $800,000 for its education coverage, which may be his way of guaranteeing he will never be exposed as a know-nothing in his hometown paper.

Now that Klein is free, she writes that miracle schools are mirages. Her case in point: Ballou High School in D.C., which claimed that all its graduates were accepted into colleges.

“It shouldn’t surprise anyone to read about another supposedly phenomenal school accomplishment that ended up being more mirage than miracle.

“The latest example comes from Washington, D.C., where in June, it was widely reported that Ballou High School, where few students tested as proficient in math or English, had nonetheless, incredibly sent all its seniors to college.

“Incredible, indeed. When NPR and the local public radio station WAMU joined forces to re-examine the Ballou miracle, they found that half of the graduates had missed at least three months of classes in a single school year. A fifth of them had been absent for more than half the school year. Teachers complained that they had been instructed to give students a grade of 50 percent on assignments they hadn’t even handed in, and that they were pressured to pass students whose work didn’t remotely merit it.

“Students complained that they were utterly unprepared for the colleges that everyone had been so proud of them for entering. And credit recovery courses – which have been criticized as too easy – played a big role in their graduations. The NCAA rejects most of these courses for college athletes; why shouldn’t colleges have the same requirements for other students?

“More than anything else, though, the Ballou High case teaches us once again that when we place intense pressure on schools to meet certain numbers, they’ll find a way to do it – one that might not involve providing a superior education. Carrots and sticks alone don’t improve schools, certainly not in the absence of funding to reduce class sizes (and teacher workloads), or to help low-income students overcome obstacles.”

Will the L.A. Times editorial board acknowledge that intense pressure to raise test scores and graduation rates corrupts not only the measure but the process being measured?

Will someone tell Eli Broad or has he surrounded himself by yes-persons?

This is Campbell’s Law, which is inexorable.

Read more here: http://www.sacbee.com/opinion/op-ed/article190028449.html#storylink=cpy

Erich Martel is a retired D.C. teacher and current whistleblower. The principal of the much-criticized Ballou High School–where chronic absenteeism was ignored–has been removed, but the deeper problems have not been addressed. He points out in a letter to the editor of the Washington Post that the system of rewards and punishments built into the system encourages gaming the system. Standardized tests are used not to identify student needs and remediate them, but to hold schools’ accountable for meeting goals.

He writes:

There is no mystery to high school students’ deficient achievement: DCPS doesn’t use standardized tests to diagnose and remediate students’ learning needs; they’re used to hold school faculties collectively accountable, while unremediated students are socially promoted. Fifty percent of principals’ IMPACT evaluation consists of “voluntarily” set performance goals, including promotion and graduation rates at the high school level. Principals face an unethical choice: honor the grades that students earned or keep their jobs. It’s time to end troubled DCPS management policies that impede teaching and learning and teach the wrong lesson on integrity.

These are the “get tough” policies installed by Michelle Rhee in the name of “reform” and kept in place by Kaya Henderson. They have produced phony test scores and phony graduation rates. This is the fruit of corporate reform, where meeting the goals matter more than truth.

Campbell’s Law takes its toll again. And always. By using the measure as the goal, both the measure and the goal are corrupted.

“The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.”

On a similar note, Campbell also wrote:

achievement tests may well be valuable indicators of general school achievement under conditions of normal teaching aimed at general competence. But when test scores become the goal of the teaching process, they both lose their value as indicators of educational status and distort the educational process in undesirable ways. (Similar biases of course surround the use of objective tests in courses or as entrance examinations.)

Last July, I wrote about a struggling high school in D.C. where 100% of the seniors graduated and were accepted by colleges. The story appeared on NPR, and I wrongly assumed that they had done fact checking. I am not a reporter, and I do not have a staff to check out claims. NPR does. But they took the claim by D.C. administrators at face value, without checking.

Now NPR reports that the original story was fishy. Better late than never.

“An investigation by WAMU and NPR has found that Ballou High School’s administration graduated dozens of students despite high rates of unexcused absences. WAMU and NPR reviewed hundreds of pages of Ballou’s attendance records, class rosters and emails after a DCPS employee shared the private documents. The documents showed that half of the graduates missed more than three months of school last year, unexcused. One in five students was absent more than present — missing more than 90 days of school.”

“According to DCPS policy, if a student misses a class 30 times, he should fail that course. Research shows that missing 10 percent of school, about two days per month, can negatively affect test scores, reduce academic growth and increase the chances a student will drop out.”

The majority of the graduating class missed more than six weeks of school.

So now we understand how the reformers in charge of the DC school system got the graduation rate up. By lowering standards. By lying.

Remember Campbell’s Law.

“The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.”

When you reward schools for higher scores, they will get higher scores, by hook or by crook. When you reward them for higher graduation rates, they will do what it takes—including lowering standards—to reach the goal.

John King, who served as Secretary of Education after Arne Duncan departed, went to the Cleveland City Club to praise high-stakes testing as the route to equity and civil rights. He spoke highly of No Child Left Behind and its successor, the federal Every Student Succeeds Act.

He is so wrong. Not just wrong, but misinformed, misguided, and ignorant of facts and evidence about the injurious effects of high-stakes testing on children, teachers, schools, and education. When you read things like this, you remember how the Obama administration sold public education out and paved the way for Betsy DeVos.

All that testing, he said, raises test scores.

Clearly, he never read the report of the National Academy of Sciences (2011) “Incentives and Test-Based Accountability in Education.”

I recommend that King read Daniel Koretz’ new book: “The Testing Charade: Pretending to Make Schools Better.” Koretz shows that high-stakes testing produces score inflation, teaching to the test, cheating, and loss of instructional time for non-tested subjects.

Someone should explain Campbell’s Law to John King. Whenever high stakes are attached to a measure, it corrupts the measure as well as the social process that is being measured. That means that when you attach high stakes to tests, you can no longer trust the test results and you mess up what is being measured.

Tests are normed on a bell curve. Every bell curve has a top half and a bottom half. The most advantaged kids cluster in the top half. The most disadvantaged kids cluster in the bottom half. Could someone explain to John King that standardized tests never produce equity? That they measure gaps without reducing them? That they discourage children who are told year after year that they didn’t meet the standard? How does it promote equity to rely on a tool that is designed to measure and reproduce inequity?

Tom Ultican left the private sector to teach physics and mathematics in a California public school.

He writes here about how setting targets for graduation rates has produced the same corruption as NCLB’s mythical target of 100% proficiency on tests. It is the inexorable workings of what is known as Campbell’s Law in social science: “The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.”

The corruption is by no means limited to California. It is nationwide, as Ultican shows.

It is promoted by schools eager to meet targets but also by for-profit entrepreneurs, who make easy money with inferior products.

The unanswered question in this discussion is what to do to help the students who don’t earn a legitimate high school diploma. Without it, they will have trouble getting a job. How do we restore the meaning of a high school diploma without leaving hundreds of thousands with no job prospects. The best answer is very likely career and technical training, especially if it is not forced into the same college-prep mold as other paths.

This is one of those brilliant posts that I am honored to share with you:

“A miracle has occurred. America’s high school graduation rates peaked at about 77% in 1970 and then drifted down for almost four decades to 69% in 2007. Astoundingly, even with increased graduation requirements rates have shot up.

“Many school districts in California now require all students to meet course requirements for entering the University of California system to graduate from high School. That is a dramatic increase in academic rigor. Yet, in 2016, over 83% of California’s freshman cohort graduated on time. In 2012, 81% of the freshman cohort in America graduated on time. These record setting numbers are the result of knuckleheaded political policy, cheating and credit recovery.

“What is Credit Recovery and Where did it Come from?

“In the 1990’s politicians like Bill Clinton and Jeb Bush were pushing for standards in education and accountability measures. Jeb Bush’s infamous school grading system called for 25% of a high school’s grade to be based on graduation rates. Bill Clinton wrote in 1998,

We have worked to raise academic standards, promote accountability, and provide greater competition and choice within the public schools, including support for a dramatic increase in charter schools.”

“We know that all students can learn to high standards, and that every school can succeed if it has clear instructional goals and high expectations for all of its students; ….”

“Donald T. Campbell’s 1976 paper presented a theory about social change that is now widely revered as Campbell’s Law: “The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.”

“Exactly as the Social Scientist, Campbell, postulated, this national push to increase the standards of school rigor and to use social indicators (graduation rates and high stakes testing) to evaluate schools has introduced distortion and corruption.

“How were school leaders going to protect their institutions and their own jobs from the ravages of horribly shortsighted and uninformed education policy? The solution was obvious; teach to the test and find a way to raise graduation rates.

“To the rescue, came both the Walton Family Foundation and the Bill and Melinda Gates Foundation along with many other lesser contributors. They perceived it was time for advancing the privatization of public education and accelerating the adoption of technology in education. Credit recovery was a perfect vehicle.”

Read on to learn about the roles of many other organizations that pushed the naive narrative that setting a goal and punishing those who didn’t reach it would produce great results.

Statisticians Mark Palko and Andrew Gelman explain why a relentless obsession with test scores ruins the value of the scores. As their prime example, they refer to Eva Moskowitz’s Success Academies, where children and teachers live for higher scores. Not only are the children’s names and ranking posted, so are the teachers’.

You remember Campbell’s Law? That’s the axiom that says when you attach consequences to a measure, the measure loses its validity.

They write:

“When a school uses selection and attrition policies that effectively filter out many of the extremely poor, students speaking English as a second language, and the learning disabled, that clearly calls into question test score advantages that such a school might have over an ordinary public school.

“But the problems run even deeper than most critics realize: A look at the data combined with some basic principles of social science suggests that the practices of no-excuses charters are undermining the very foundation of data-based education reform.

“As statisticians with experience teaching at the high school and college level, we recognize a familiar problem: A test that overshadows the ultimate outcomes it is intended to measure turns into an invalid test.

“Back in the old Soviet Union, factories would produce masses of unusable products as a result of competition to meet unrealistic production quotas. Analogously, many charter schools, under pressure to deliver unrealistic gains in test scores, are contorting themselves to get the numbers they’ve promised. They’re being rewarded for doing so. But that monomaniacal focus on test scores undermines the correlation between test scores and academic accomplishment that originally existed.”

They note that Success Academy has astonishingly high test scores, yet for two years in a row, not a single one of their eighth grade students won admission to one of the city’s elite high schools. In the third year, some did (11% of those who took the test from SA).

In a comment on this post, Gary Rubinstein (a blogger who teaches at Stuyvesant High School, an exam school) writes:

“One thing to note, the 11% specialized HS acceptance rate–6 out of 54–is inflated since there were 200 kids who feasibly could have sat for that test but only 54 did.” Of 200 students at Success Academy who were eligible to take the test, 54 did, and 6 gained admission.

It is better to have high scores than low scores, but they should never be the measure of teacher quality or school quality. Making them too important ruins their value.