The imperative for evidence-based funding and policymaking in education, and the role of evidence in the EOF model
A new paper by the Education Outcomes Fund's (EOF) Chief Programs Officer, Jared Lee, published in the Stanford Social Innovation Review argues for the need for greater use of evidence to inform funding and policy decisions following a global review of education program evaluations, that demonstrated that only around half of the programs had any impact on learning outcomes at all, and of those that did, the most cost-effective interventions had an order of magnitude greater impact per dollar spent than the rest.
As outlined in Figure 1, around half of these studies found zero or negative impact on learning and were therefore not cost-effective by definition. Of those that had a positive impact, the weighted average impact per child per dollar spent of those in the top quintile was ~100 times greater than the impact of the lowest quintile, and ~9 times greater than the median of the available studies.
Figure 1: Impact on learning, from all program evaluations with impact and cost-effectiveness data
The paper highlights several important conclusions that can be drawn from this data:
1. Seemingly thoughtfully designed programs can have counterintuitive results, and so it is critical to be guided by the evidence as to what is actually working, not what we think will work.
2. Whether or not government and donors succeed in identifying and supporting the most cost-effective programs in each local context will dwarf any efficiency improvements that could be made in other areas that often garner significantly more attention, such as squeezing out modest savings from procurement or program management costs.
3. There is an enormous opportunity to improve the average ‘bang for our buck’ of our investments in education systems. Doing so will be absolutely critical if we are to have any chance of addressing the global learning crisis, particularly given the enormous scale of the financing gap.
The paper outlines the generation and use of evidence as an often-overlooked but central rationale for outcomes-based funding. It demonstrates the use of evidence throughout the EOF model as can be seen in Figure 2.
Figure 2: Illustrative use of evidence through the EOF program lifecycle (role of evidence in orange)
The paper concludes that "in the debate about the advantages and limitations of impact bonds and outcomes funds, policymakers should recognize the central role of evidence in these instruments."
The full paper can be found at the Stanford Social Innovation review here, and the appendix and additional diagrams are below.
Appendix: Interventions considered by the Global Education Evidence Advisory Panel and EOF
The paper refers to and builds upon research compiled by the Global Education Evidence Advisory Panel (GEEAP). Figure 3 below is an extract from the “Smart Buys” policy note, illustrating on a log scale the impact on Learning Adjusted Years of Schooling of 150 interventions in education across 46 countries, of which 71 had recorded cost-effectiveness data available. Further analysis of these can be found in a background academic paper (Angrist et al. 2020).
For the analysis in the paper, the authors considered all intervention studies from the GEEAP that reported cost-effectiveness data. Figure 4 lists those that had a positive impact, and how they compared on cost-effectiveness. Interventions are grouped by quintile in order to illustrate the general pattern that is shown in Figure 1.
Figure 3: Learning-Adjusted Years of School (LAYS) gained per $100, by intervention type
Figure 4: Interventions with cost-effectiveness data that showed a positive impact
