Sarah Reckhow of Michigan State University University and Megan Tompkins-Stange of the University of Michigan studied the ways in which foundations fund research that advances policies they believe in. They use the issue of teacher quality, specifically, to demonstrate how the Gates Foundation and the Broad Foundation underwrote research that provided evidence for evaluating teachers by the test scores of their students (VAM, or value-added modeling). The research supported a policy that the Obama administration wanted to implement.
VAM turned out to be highly ineffective and demoralized teachers, but the big foundations gave the Obama administration the back-up the administration needed for their demand that teachers be evaluated by their students’ test scores. The American Statistical Association warned that VAM was an invalid measure of individual teachers, as did other scholarly and professional organizations, but Obama and Duncan ignored the naysayers.
Reckhow and Tompkins-Stange write:
After the Obama Administration took office in 2009, a number of former Gates Foundation officials assumed senior roles in the Department of Education under Secretary Arne Duncan, and were influential in drafting Race to the Top, a $4.3 billion competitive grant program designed to induce states to comply with specific policy reforms, including the use of value-added methods in evaluation programs. The Department of Education’s call for proposals stated that Race to the Top grant winners would focus on advancing four specific reforms:
“Adopting standards and assessments that prepare students to succeed in college and the workplace and to compete in the global economy; building data systems that measure student growth and success, and inform teachers and principals about how they can improve instruction; Recruiting, developing, rewarding, and retaining effective teachers and principals, especially where they are needed most; and turning around our lowest-achieving schools.”
These implicit and explicit references to value-added measures and the need to evaluate and compensate teachers based on their effectiveness are evidence of the emergent debates around using student test scores to determine teacher pay—another plank of the education reformers’ theory of change. An interviewee from a foundation commented on the fact that after Race to the Top, states were required to “put together evaluation systems for teachers and states would begin to link this to hiring and firing.” The fact that this particular reform had acquired such political capital in a relatively short time was, in the words of this interviewee, “remarkable.”
Creating an evidence base
In addition to maintaining close networks with policy elites, foundations actively engaged in commissioning original research designed to provide an evidence base relevant to their policy priorities. Foundations make grants to intermediary organizations to conduct “advocacy research,” which has the explicit objective of being injected into policy discourse to be cited as empirical justification for desired reforms (Lubienski et al. 2009). Unlike traditional peer-reviewed research, which may pose uncertain conclusions regarding policy implications, advocacy research is shaped by specific policy objectives and political strategy and is typically produced by think tanks and nonprofit organizations, rather than universities (Shaker and Heilman 2004). The level of empirical rigor in advocacy research exists on a spectrum, from employing highly rigorous methods and considerations of external and internal validity, to omitting discussion of methods entirely.
While foundation-funded advocacy research is by no means the only source of policy-influential research in the teacher quality debate, it is central in Congressional hearings during our study period. Between 2000 and 2016, only nine research reports were cited three or more times by witnesses (and only one of which was peer-reviewed). The fourth-most cited report, which was consistently referenced in our interviews, was a 2009 advocacy research report by The New Teacher Project entitled The Widget Effect—a call to arms about the need for systematic teacher evaluation systems in order to distinguish between low-quality and high-quality teachers using test score-based evaluation methods. The report stated that “institutional indifference to variations in teacher performance” resulted in systems that perpetuated low-quality teaching across the country, taking aim at evaluation systems that relied predominantly on observational meth-ods as opposed to econometric approaches (Weisberg et al. 2009). Several education reform-oriented foundations including the Gates Foundation, Walton Family Foundation, Robertson Foundation, and Joyce Foundation funded the report. Within a month of its release in 2009, Secretary Duncan made the following statement about the report in a speech:
“These policies…have produced an industrial factory model of education that treats all teachers like interchangeable widgets. A recent report from the New Teacher Project found that almost all teachers are rated the same. Who in their right mind really believes that? We need to work together to change this.
The Widget Effect was praised by many interviewees as a triumph of advocacy research—a clear proposal and message, presented in a comprehensible and digestible format, that made a complicated issue more palatable. More importantly, however, the report was also a triumph for the policy networks surround-ing teacher quality discourse—within a month, the report had had such impact that Secretary Duncan was referencing it in major speeches, which was accomplished by disseminating it through policy networks among actors with shared preferences.
The widespread recognition of The Widget Effect was emblematic of the rising prominence of advocacy research in policy debates. In the last ten years, education policy scholars have observed a shift toward targeted advocacy research funded by foundations, particularly surrounding issues of market-based policy interventions (Henig 2009; Lubienski et al. 2009). Contemporary examples of advocacy research contest the traditional conceptualization of expert researchers being separate and distinct from politics. According to Kingdon (2011, p. 228):
“The policy community concentrates on matters like technical detail, cost-benefit analyses, gathering data, conducting studies, and honing proposals. The political people, by contrast, paint with a broad brush, are involved in many more issue areas than the policy people are, and concentrate on winning elections, promoting parties, and mobilizing support in the larger pol-ity.”
In current education policy networks, however, the converse is true, as researchers and advocates may overlap. One interviewee, a staff member of an education advocacy organization, described her role on a Gates Foundation-funded advocacy research project: “We saw a need to be more involved, not just from putting ideas out there but to help guide the conversation more hands-on.” Foundations, particularly those that endorse common education reform priorities, are now more likely to reject the traditional model of funding basic research in universities intended for diffusion into policy networks, but without the added leverage of a dedicated marketing structure to ensure, rather than impute, that the research reaches its intended audience.This is particularly true for foundations that identify as strategic philanthropies who are more likely to assertively use research as a tool to advance their policy goals. Strategic philanthropy is structured around the managerial concept of strategic planning, emphasizing clearly articulated goals from the outset of an initiative, the use of research to substantiate decisions, accountability from grantees in the form of benchmarks and deliverables as measured in incremental outcomes, and evaluation to assess progress toward milestones (Brest and Harvey 2008).
Strategic funders also prioritize measurable returns on investments. Applying this formulation, basic research can appear very costly, with high levels of uncertainty or ambiguous returns, while targeted advocacy research promises better yield.Interviewees described strategic foundations—most notably, the Gates and Broad Foundations—as highly influential leaders within the teacher qual-ity policy network and depicted foundations’ theory of change as based on the assumption that teacher evaluation was necessary to advance other education reform goals, such as pay for performance and alternative teacher certifications. They also focused on these foundations’ use of research evidence as political in nature, departing from the “expert-led model of change” that Clemens and Lee (2010) describe and moving toward a model wherein researchers and advo-cates pursued similar goals: to inject policy ideas into political discourse more directly than their traditional philanthropic approaches.
The authors go on to describe the Gates Foundation’s big investment in the MET program (Measures of Effective Teaching). As several interviewees comment, the research started out with a desired outcome, then sought the evidence to back it.
The research paper was published in 2018 and remains timely.
What we don’t know yet is whether the Gates Foundation learned anything from its multiple failures in the field of education.
Diane Ravitch's blog 