Harvard Magazine published a noteworthy article about the failure of pay-for-performance plans in hospitals, written by Marina N. Bolotnikova. It is worth your time to read. It is devastating.
The logic of pay-for-performance systems is simple enough: pay doctors and hospitals based on how well their patients are doing, rather than on the number of medical services they provide. The payment structure was designed to fix a central problem in American healthcare. The United States spends far more per person on healthcare than any other country, yet has the poorest health outcomes in the advanced world. Pay-for-performance, also known as value-based purchasing, was meant to encourage doctors to optimize the welfare of patients while discouraging spending on unnecessary care.
There was never any evidence that pay-for-performance works, said Li professor of international health Ashish Jha, who investigates the effectiveness of economic incentives in the healthcare market. A recent paper published this April in the BMJ by Jha finds that the federal pay-for-performance program under Medicare, the Hospital Value-Based Purchasing (HVBP) program, hasn’t had any impact on mortality rates. Nor do HVBP-participating programs show any statistically significant advances over the small number of hospitals that don’t participate in the program. No hospitals, not even those with the worst mortality rates before the program was implemented, showed an improvement that could be attributed to pay-for-performance. “We looked across all different conditions and couldn’t find a single one where it seemed to have a meaningful effect,” Jha explained.
In the early 2000s, the Bush administration ran a pilot version of pay-for-performance at about 200 hospitals that agreed to tie their payments to certain quality measures. Even then, Jha said, “the broad consensus in the community was that it didn’t really work.” Despite this, the system was implemented on a national scale by the Affordable Care Act (ACA), which created the HVBP program at more than 90 percent of U.S. hospitals. “There was a sense that we needed to begin somewhere,” he explained, “and that this program would be good to test out nationally.”
Medicine and education could learn from one another. Why has there been so little attention to the persistent failure of pay-for-performance plans in education? States and districts and the federal government pour hundreds of millions, billions, into developing incentives, despite the fact that the National Academy of Sciences studied the issue in education and said that such plans don’t work, especially when the bonuses are tied to test scores.
Diane Ravitch's blog 
David Berliner already documented the issues with this in one of his books. Nothing new, just like a century’s worth of research & experiments with merit pay in education that never works.
LikeLike
Capitalism is a disease that societies get.
We can only hope that democracy will recover, but the prognosis is not good.
LikeLike
Pay for performance sounds logical at first, but it does not make sense for education or healthcare. Both fields are human services, and pay for performance starts off with the assumption that both groups are under performing, and a bonus will incentivize the workers’ performance. Both healthcare and education deal with lots of variables that cannot be controlled by the service provider. In healthcare doctors are not responsible for the patients’ general health and the severity of the presented condition. When the pencil pushers try to create a performance algorithm, they make lots of one size fits all assumptions that may or may not be true in order to generate a formula. Hospitals and doctors that take on the most difficult cases or operate in poor areas get penalized. In education schools and teachers that serve the poor, special needs, at-risk, and ELLs students all get penalized in the same manner as some doctors and hospitals. Economists should work on improving the economy as there is lots of work to be done there, and leave healthcare and teaching to professionals in those fields. By the way if we used pay for performance for members of Congress, lots of them would barely be able to cover their Starbucks’ bill.
LikeLiked by 1 person
“and pay for performance starts off with the assumption that both groups are under performing,”
But the absolute hubris of that- the assumption that people OUTSIDE health care and education can judge “under-performing” and should blow up systems and “recreate” them- where does THAT come from?
LikeLiked by 1 person
You’ve hit the nail on the head!
LikeLike
“Medicine and education could learn from one another”
Absolutely they could. I’m so glad you’re making this connection because the complaints of health care people on “data-based” systems sound a lot like the complaints of teachers.
They don’t want their profession run by business people and economists. They want it run by health care experts.
I think there’a a real equity question for ed reformers, too. We have a privatized health care system. Tons of public funding goes to private entities. Is it “more equitable” than a public system? Is it more efficient? Why would K-12 education be more equitable and efficient if it were privatized?
Someone could probably do a really interesting study on when everyone decided economists should run everything in the world. When did these people become experts on every single sector and public policy? Why do they have such outsized clout? Economists haven’t even been very good at predicting the economy. Why does everyone in DC defer to them?
LikeLike
It still boggles my mind that DC “fixed” the private health care system (which wasn’t working) and at the same time started work on turning K-12 public education INTO the private health care system.
“The privatized health care system is an inequitable, expensive mess. Let’s do that to K-12 public education!”
What are they thinking? They think privatizing schools will work out better?
LikeLike
I actually did hear a couple of years ago (although now I can’t remember the source) that Depts. of Economics were turning out too many labor economists for them to be hired in traditional fields. That is what they started “offering their services” to other sectors. Has anyone else heard something similar?
LikeLike
Sadly, with both hospitals and education, these kind of bonuses just incentivize hospitals and schools to use whatever means necessary to get rid of the patients and children who are harmful to their overall results.
If you go into the hospital for a simple appendectomy and you feel well-served by the staff, you are thrilled. If you are diagnosed with stage one cancer and need a course of chemo, you are thrilled. It is only when your child gets advanced cancer and you realize the same hospital staff that wooed your appendectomy is doing whatever necessary to convince you your child will feel misery and pain until you find a different hospital that you suddenly realize who is paying the price for that nice appendectomy you received. The sickest and most expensive patients.
And if you are lucky and your family remains healthy, you can pretend that it’s all okay.
LikeLike
From the ed reform echo chamber files:
“But magnet applicants face long odds and long waitlists. Magnet programs received more than 44,000 applications for only 8,700 total available seats in the upcoming school year, according to district data from May. For years, the difficulty of getting a spot has prompted calls from district leaders — including, recently, King — to expand L.A. Unified’s magnet programs.”
There are public schools with wait lists. Yet, no one in ed reform works on expanding them.
Gosh, I wonder why not? They say they’re “agnostics”. One would think there would be some effort to expand these schools. It’s “all about the kids” right?
http://www.scpr.org/news/2016/07/15/62634/la-unified-s-magnet-school-programs-in-demand-but/
LikeLike
Great point! “Reformers” don’t clamor for more magnet seats because there is no union to bust or profit to be extracted.
LikeLike
Wait, I thought this blog was a fan of “multiple measures.” Is that still true? Mortality rates are one measure of performance, but there are others, too. The literature reaches more varied conclusions when you consider these other measures. It’s not a slam-dunk case for pay for performance, but it also doesn’t mean it is never a good idea.
Notice, too, that the conclusion of the study’s author is not to completely reject pay for performance, but to do it better. Diane strategically cuts off the quote from the article right before this paragraph, which undercuts the premise of Diane’s post: ” Now, a few years into the ACA, Jha’s research measures the disappointing results of pay-for-performance. He blames the program’s failure on its poor design: HVBP affects only up to 2 percent of Medicare payments to hospitals, and the size of the financial incentive is determined through a complex formula with many variables, some of which doctors might not be able to control. As Jha put it: “Imagine if your boss came to you and said, ‘I’ll put just 2 percent of your salary at risk, and you have to do these 38 things.’ So you just ignore it.” Clearer and larger incentives, he argued, might begin to produce results. “Instead of incentivizing 35 things, incentivize three things.” ”
Perhaps the best quote from the article is the important admonition that whenever performance is accounted for it should be done with a deep understanding of implementation and ground-level practice. That makes a ton of sense.
LikeLike
Paul,
I never said I favor multiple measures or single measures. I oppose incentives for professionals. Read the National Academy of Sciences report thatch Test-Based Incentives. Read Edward Deci on incentives. Read Dan Ariely on incentives. Read Deming on incentives. They are counterproductive and they undermine morale among professionals. We were supposed to have moved beyond Taylorism.
There is no merit pay program that has ever proved successful.
LikeLike
Do you think that non-experts in medicine or education can have a deep understanding of these fields? I am opposed to the neo-liberal agenda, mostly because I see the impact of what it brings, but I would never have the hubris to claim I have a deep understanding of it. Economists should stay in their own lane! They cannot generate meaningful data from false assumptions and and superficial understanding of the issues. Not everything can or should be distilled into a number!
LikeLike
Part of this depends on how the incentives are applied, right? So I totally agree with you—in education, especially—that mindlessly thinking that applying teacher-pay bonuses to people based on test scores is a bad idea. Pits individual teachers against each other, potentially. Undermines culture. Yes. But incentives that reward or celebrate entire organizations for their performance, helping to foster teamwork, can provide a middle ground, I think. That’s what smart uses of incentives try to do. They try to harness the entire energies of staff to make better choices about how to use the organization’s resources, with the incentive being that if its is done well then the organization gains greater autonomy, among other things.
LikeLike
Incentives may work to help people reach sales quotas or sell more products. It doesn’t work in education or health care because these are human services and because of all the variables outlined in the posts above. Education and medicine are a lot more complex.
LikeLiked by 1 person
Organizational incentives also have issues. Stock options used to be popular, but the organizations become so focused on stock price and short term gains, that a balanced approach is lost. Who would dare risk a project that has potential long term gains but short term losses affecting the price at the closing bell? Plus, like the Donald says, the system is rigged and rarely do rank and file benefit from meager options dangled like a far off, 5 year vested carrot.
LikeLike
How about merit pay for corporate executives. They wouldn’t get to give themselves bonuses while we give their companies bailouts.
LikeLike
How about merit pay for economists. They would get paid nothing. They contribute nothing of any value. They’re never right about anything.
LikeLike
Your comment about economists (I’m not one, by the way) is the definition of throwing out the baby with the bathwater.
Helen Ladd is never right?
http://www.nytimes.com/2011/12/12/opinion/the-unaddressed-link-between-poverty-and-education.html
Jesse Rothstein?
Click to access 20160316-SchoolFinanceReform.pdf
Please don’t reply by simply saying I have mentioned only two names. Read the papers of these economists and you will see dozens of others cited.
LikeLike
Paul Manna,
I know and respect both Helen Ladd and Jesse Rothstein.
But it is also true that economists like Eric Hanushek (whom I also know well), Tom Kane, Raj Chetty, Jonah Rockoff, and their associates have helped to create the current punitive atmosphere in education by trying to quantify education in relation only to test scores. They have sold policymakers on the spurious notion that the best teachers “produce” the highest test score gains, and those who don’t should be fired, “sooner rather than later,” as one of Chetty’s colleagues told the NY Times. Every debate with them descends solely to quantitative measurement, which educators know is neither fair nor appropriate.
LikeLike
If you read Five Days at Memorial by Sheri Fink about a hospital in NOLA during and immediately after Hurricane Katrina, you will be amazed at the parallels between health care and education. The first 100 pages of the book outline the pressure put on the health care industry prior to this emergency and are eerily similar to the pressures on public education today. Spoiler alert – business people are more interested in profits than providing adequate health care.
LikeLike
Here’s an article from the 1/2012 Harvard Business Review on the well known failures of tying pay to performance. https://hbr.org/2012/01/tackling-business-problems/ar/1#.TvtyhA4E5ps.twitter
LikeLike
The two articles below the one on performance pay are also worth a read, especially the one on evaluations.
LikeLike
Here is a more general article by Bruce Baker that touches on merit pay while examining why elite private schools engage in virtually none of the so called business model improvements of reform. Baker makes the hypocrisy of reformers quite clear vis a vis their well informed choices for their own kids vs. what they choose to impose on other peoples children. https://schoolfinance101.wordpress.com/2012/02/17/borrowing-wise-words-from-those-truly-market-based-private-independent-schools/
LikeLike
Speaking of merit pay, Obama hopes to invest in it big time, IF….his FY2017 budget is funded. http://www.ed.gov/news/press-releases/president-obamas-2017-budget-seeks-expand-educational-opportunity-all-students
He has called for a $1 billion mandatory investment to support– “RESPECT: Best Job in the World” program. RESPECT stands for “Recognizing Education Success, Professional Excellence, and Collaborative Teaching. This is another try at a 2010 McKinsey & Co. vision of teaching pushed into a federal agenda by a series of “conversations” lead by USDE and surrogates for several years, as outlined here. https://www.ed.gov/teaching/respect-conversations
In 2012, Obama envisioned a $5 billion program of competitive grants to “transform “ teaching. Judging from the FY 2012 budget, Congress did not fund the RESPECT program but it did fund aspects of the program—discretionary funding for: ” Effective teachers and leaders State grants for $2,500,000,000; Teacher and leader innovation fund for $500,000,000; and Teacher and leader pathways for $250,000,000.
The RESPECT program is basically a pay-for-performance plan from McKinsey & Co. with multiple tiers of compensation for “effective” teachers, no job security, but increasing compensation for more responsibilities.
USDE has pushed the policies, misleadingly called RESPECT, by a process of convening groups and leading them to a “consensus“ on the need to ”transform” the teaching profession. The process is not different from building a “consensus” about the Common Core, forged by aligning state standards to a preconceived set of standards, then marketing the result as a consensus and “state-led.”
The RESPECT project is certainly marketed as if it rests on a 2012 “consensus document” for “transforming” the profession of teaching. That document was organized around three criteria for “success” and “seven core elements of a transformed profession.
The 2012 consensus document begins by asserting that:
Success in transforming the teaching profession is to be judged “by our students’ outcomes.” Three are named.
“1. High levels of student achievement, judged by multiple measures that assess students’ ability to understand and apply the knowledge and skills that matter most to their readiness for college, careers, and citizenship;
2. Increased equity, judged by continuously narrowing the gaps in achievement and opportunity between more and less privileged populations of students; and
3. Increased global competitiveness, judged by American students’ academic performance on internationally benchmarked measures.”
There is not much new there.
The document includes seven “core elements” of a transformed profession.These are one-paragraph statements filled with jargon calculated to portray the work of teachers as a matter of recruiting talent, plenty of on the job training, the (endless) pursuit of continuous improvement, and with proper workforce management achieved by evaluations tied to compensation and opportunities for advancement (more work for higher pay).
This jargon-filled statement was signed by officers of: The American Association of School Administrators, American Federation of Teachers, Council of Chief State School Officers, Council of Great City Schools, Federal Mediation and Conciliation Services, National Education Association, National School Boards Association, and U.S. Department of Education.
The “core elements are general.” But by 2013, these signators are treated as if they have endorsed detailed proposals in the another version of RESPECT– a nearly perfect match with the original 2010 McKinsey & Co. “white paper” with line-by-line talking points to promote this “transformative” agenda. http://www.ed.gov/teaching/national-conversation
This is also to say that the AFT and NEA and others who signed on to the consensus document were either clueless about the plan to kill collective bargaining or they have, in fact, endorsed the “RESPECT Blueprint proposed by McKinsey & Co. tweaked by Duncan and USDE hires, and left as a legacy by Obama, as a parting shot at teachers.
The RESPECT project, in all iterations, has been a sales pitch for an agenda designed to create a national churn in the teacher workforce, with standardized measures of performance, and more talk about tbenifites for teacher autonomy than are actually permitted by the whole scheme. See the overall “Blueprint” with a diagram on page 23 and important appendices here: http://www2.ed.gov/documents/respect/blueprint-for-respect.pdf
The bottom line and parting shot to teachers from Obama is this: Forget collective bargaining, forget any escape from annual measures of “effective teaching,” forget tenure unless you can survive the five to seven years wait-time and also sustain ratings of being highly effective for the rest of your career.
That means you must be at the top of the heap for three out every five years to have a long-lived career, or you are dumped, fired, or go back down the career ladder and start again. The Blueprint calls for use of the same old “multiple measures”of “effective” and “highly effective” teachers including student academic growth, the euphemism for VAM and SLOs.
Oh, forgot to mention, you enter teaching as a one or two-year“resident” at a salary of $20,000. How is that for respect? If, in two years, you do not make the cut for the next step up the ladder, as a ”beginning teacher,” you are done.
LikeLike
Some have a religious like belief that capitalism can solve everything. Our Constitution, of course, doesn’t support that but they want to believe that it does.
LikeLike