Jerry Z. Muller wrote a warning about how data-driven organizations can distort their own goals and purposes.

In education, we have known about the dangers of incentives for test scores for a long time. In 1976, sociologist Donald Campbell that “the more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social process it is intended to monitor.” When the tests assume too much importance, there will be cheating, gaming the system, narrowing the curriculum, and other unwanted consequences.

A dozen years ago, Richard Rothstein wrote an excellent paper called “Holding Accountability to Account,” showing how incentives can perversely affect and undermine the goal that are sought (it is free on the internet).

In 1990k Andrea A. Gabor wrote a book about W. Edwards Deming called The Man Who Discovered Quality, in which she explained Deming’s contempt for merit pay and bonuses, which cause employees to think about themselves and not about the organization and its larger purposes.

Muller wrote a recent article about “metric fixation” in which he reviewed the flaws of data-driven work:

More and more companies, government agencies, educational institutions and philanthropic organizations are today in the grip of a new phenomenon. I’ve termed it ‘metric fixation’. The key components of metric fixation are the belief that it is possible – and desirable – to replace professional judgment (acquired through personal experience and talent) with numerical indicators of comparative performance based upon standardized data (metrics); and that the best way to motivate people within these organizations is by attaching rewards and penalties to their measured performance. 

The rewards can be monetary, in the form of pay for performance, say, or reputational, in the form of college rankings, hospital ratings, surgical report cards and so on. But the most dramatic negative effect of metric fixation is its propensity to incentivize gaming: that is, encouraging professionals to maximize the metrics in ways that are at odds with the larger purpose of the organization. If the rate of major crimes in a district becomes the metric according to which police officers are promoted, then some officers will respond by simply not recording crimes or downgrading them from major offences to misdemeanours. Or take the case of surgeons. When the metrics of success and failure are made public – affecting their reputation and income – some surgeons will improve their metric scores by refusing to operate on patients with more complex problems, whose surgical outcomes are more likely to be negative. Who suffers? The patients who don’t get operated upon.

When reward is tied to measured performance, metric fixation invites just this sort of gaming. But metric fixation also leads to a variety of more subtle unintended negative consequences. These include goal displacement, which comes in many varieties: when performance is judged by a few measures, and the stakes are high (keeping one’s job, getting a pay rise or raising the stock price at the time that stock options are vested), people focus on satisfying those measures – often at the expense of other, more important organizational goals that are not measured. The best-known example is ‘teaching to the test’, a widespread phenomenon that has distorted primary and secondary education in the United States since the adoption of the No Child Left Behind Act of 2001.

How many times do we have to hear the same advice and ignoring it?