What Makes Educational Interventions Stick? Teaching the Right Skills in the Right Environments  

PSPI Live is a 60-minute live event series featuring presentations by the authors of articles published in the APS journal Psychological Science in the Public Interest (PSPI), along with perspectives and reflections by other authors, policymakers, or representatives of important stakeholder groups.

Time-limited educational interventions, such as an extra reading course or a motivational intervention, may have long-lasting effects on a person’s life trajectory, but sometimes their effects are short-lived. Understanding the factors that influence and contribute to the persistence and fade-out of interventions can improve theories of human development and help to create meaningful interventions, with implications for practice and policy.  

A PSPI Live on October 12 explored these phenomena, with perspectives from Drew H. Bailey (University of California, Irvine) and David S. Yeager (University of Texas at Austin), coauthors of “Persistence and Fade-Out of Educational Intervention Effects: Mechanisms and Potential Solutions” (Bailey, Duncan, Cunha, Foorman, & Yeager, 2020), along with Mark Schneider, Director of the Institute of Education Sciences. APS Fellow Nora Newcombe (Temple University), editor of PSPI, hosted the session. 

Start with definitions 

• Fade-out refers to declining treatment effects on measures of the same construct after the end of treatment. For example, in Florida, a policy that required middle school students who scored below a threshold in a reading test to take a remedial course improved these students’ reading scores in the year they took the remedial course. But two years later, the students performed at the level predicted by their initial scores, indicating that the intervention’s effects had faded out (Özek, 2021). 
• Persistence refers to when initial effects persist on the same outcome over the long term. When Chicago 8th-grade students who scored below a threshold in a math test took one extra period of algebra in 9th grade, this double-dose algebra intervention persistently improved their math test scores in 9th, 10th, and 11th grades above what would have been expected given their 8th-grade scores (Cortes, Goodman, & Nomi, 2015). 

• Emergence refers to when impacts on later outcomes occur in the absence of observed impacts over the medium run. In Florida, the impact of the remedial course faded out, but students who took it were more likely than peers who did not take the course to later enroll in and graduate from college (Özek, 2021). 

What the literature shows 

Looking at studies that researched the short- and long-term effects of educational interventions and identifying causal relationships, Bailey and colleagues (2020) found that 

• fade-out is ubiquitous; 

• fade-out, persistence, and emergence frequently co-occur; 
• the prevalence of persistence is difficult to evaluate because of the studies’ characteristics, including selective follow-up and reporting.  
• intensive contextual interventions (e.g., adoption, high-quality preschool for socioeconomically disadvantaged children) appear to show persistence of effects. 

What works?  

Interventions are most likely to provide persistent benefits when they target 

• the right kinds of skills—that is, skills that are malleable, fundamental (e.g., basic reading and basic math), and unlikely to develop in the absence of the intervention; 
• the right kinds of environments—that is, environments of sufficient quality to sustain growth, such as high-quality schools. However, the evidence about sustaining environments is mixed, and more studies are needed to investigate both how different environments can interact with each other, modifying their influences, and why the conclusion “good amplifies good” might be too simple.  

Implications for researchers and funders 

Future research on the effects of educational interventions should 

• have longer follow-up periods; 
• carefully measure the target outcomes and other broader measures of skills; 
• use forecast methods to evaluate what effects interventions might have in the long-term (e.g., educational attainment, earnings); 
• examine spillovers at scale (e.g., it appears that there’s a spillover effect of having highly educated peers, but psychological scientists have not studied these effects). 

• address the “social-policy dark matter” problem—investigate the factors that lead to interventions’ long-term effects, despite the fade-out of their effects over the short and medium term;  
• provide more systematic funding of long-term follow-up studies to help determine how commonly interventions persist. 

Pathways to persistence 

The persistence of educational interventions depends on multiple factors, but the interventions’ contexts, including institutional constraints and opportunities, are important mediators. David Yeager gave examples of pathways to persistence based on his own research and discussed the importance of contexts and institutional circumstances. 

• A short (20–30 min), online, self-administered intervention to foster a growth mindset (the belief that one can improve one’s skills) improved high school graduation rates four years later for students of color and low-achieving white students, with a 50% reduction in achievement disparities (Yeager et al., 2019).  
• Sustaining environments can have a positive effect; for example, having teachers with growth mindsets might determine whether the effects of interventions aimed at fostering growth mindsets persist (Yeager et al., 2022). 
• “For an intervention to last, you need both a quality seed (a well-crafted mindset intervention) and a conducive soil in which that seed can grow (contextual affordances congruent with that mindset),” Yeager wrote in a recent article in Current Directions in Psychological Science (Walton & Yeager, 2020). 
• This usually reflects institutional constraints and opportunities, such as financial aid or public prekindergarten.  
• Future research should address the effectiveness of interventions touching individuals and those touching contexts. 

The policy perspective 

What data systems are needed to track student outcomes longitudinally and assess the fade-out and persistence of educational interventions? And what challenges might exist? Policymakers interested in shoring up understanding and efficacy of educational interventions should consider 

• the immensity and complexity of the American educational system—there are 14,000 school districts, with very diversified characteristics (e.g., some are in large metropolitan areas and others are in rural areas); 
• the heterogeneity of the American population, which makes research on the fade-out and persistence of educational interventions even more complex; 

• the impact of school boards in determining researchers’ ability to get access to students; 
• how to build trust among parents, teachers, and administrators in researchers and data collectors, in the quality of the data, and that the data collected will be protected from hacking; 
• who the audience is—sharing results with parents, teachers, and school administrators helps build trust and credibility but researchers and policymakers must disseminate findings that are actionable for their audience;  
• the problematic nature of predictive analytics—algorithms and AI systems can make strong predictions, but researchers must make sure these predictions are accurate, fair, and ethical (e.g., algorithms should not discriminate people on the basis of race); 
• how to develop a better culture in schools, funding agencies, and government agencies so that they understand data and ask the right questions about how those data can be used.  

Q&A from the live session 

References 

Bailey, D. H., Duncan, G. J., Cunha, F., Foorman, B. R., & Yeager, D. S. (2020). Persistence and fade-out of educational-intervention effects: Mechanisms and potential solutions. Psychological Science in the Public Interest, 21(2), 55–97. https://doi.org/10.1177/1529100620915848 

Cortes, K. E., Goodman, J. S., & Nomi, T. (2015). Intensive math instruction and educational attainment. Long-run impacts of double-dose algebra. Journal of Human Resources, 50(1), 108-158. https://doi.org/10.3368/jhr.50.1.108 

Özek, U. (2021). The effects of middle school remediation on postsecondary success: Regression discontinuity evidence from Florida. Journal of Public Economics, 203, Article 104518. https://doi.org/10.1016/j.jpubeco.2021.104518 

Walton, G. M., & Yeager, D. S. (2020). Seed and soil: Psychological affordances in contexts help to explain where wise interventions succeed or fail. Current Directions in Psychological Science, 29(3), 219–226. https://doi.org/10.1177/0963721420904453 

Yeager, D. S., Carroll, J. M., Buontempo, J., Cimpian, A., Woody, S., Crosnoe, R., Muller, C., Murray, J., Mhatre, P., Kersting, N., Hulleman, C., Kudym, M., Murphy, M., Duckworth, A. L., Walton, G. M., & Dweck, C. S. (2022). Teacher mindsets help explain where a growth-mindset intervention does and doesn’t work. Psychological Science, 33(1), 18–32. https://doi.org/10.1177/09567976211028984 

Yeager, D. S., Hanselman, P., Walton, G. M., Murray, J. S., Crosnoe, R., Muller, C., Tipton, T., Schneider, B., Hulleman, C. S., Hinojosa, C. P., Paunesku, D., Romero, C., Flint, K., Roberts, A., Trott, J., Iachan, R., Buontempo, J., Yang, S. M., Carvalho, C. M., … Dweck, C. S. (2019). A national experiment reveals where a growth mindset improves achievement. Nature, 573(7774), 364–369. https://doi.org/10.1038/s41586-019-1466-y

Psychological Science and Education >

---