Publication bias inflates estimates of the effectiveness of common educational interventions

Bastian Andreas Betthäuser, Bartholomew Konechni, Vanessa Astrid Wittemann, Gaia Grassi, Susan Swingler, Anders Bach-Mortensen, Niklas Ayris and Una Oljaca
Additional contact information
Bastian Andreas Betthäuser: Sciences Po, Paris

No rf8kx_v2, SocArXiv from Center for Open Science

Abstract: While there is a large body of evidence on the effectiveness of educational interventions, the robustness of this evidence has not been systematically assessed. Here we conduct a preregistered systematic review, quality appraisal, and meta-analysis of all available estimates of the effects of four common supplementary learning interventions—tutoring, summer programs, afterschool programs, and extended instruction time—on children’s learning in mathematics and reading. We use multiple diagnostic techniques to assess the risk of bias of individual effect size estimates and to test for publication bias in the evidence base on supplementary learning interventions as a whole. We find that about half of all effect size estimates are at serious or critical risk of bias, with confounding and deviation from intended treatment being the most prominent risk-of-bias domains. We find substantial publication bias in the existing evidence base, indicated by a marked discontinuity in the distribution of test statistics at the conventional significance threshold and a significant association between magnitude and precision of effect size estimates. We combine five bias-correction techniques to estimate bias-adjusted effects for the four intervention types. Correcting for publication bias substantially reduces effect size estimates for tutoring, summer programs, and extended instruction time and renders estimates for afterschool programs statistically insignificant. After correction, tutoring is the only intervention that consistently yields positive and statistically significant effects. Our findings suggest that the existing evidence base substantially overestimates the effectiveness of commonly implemented supplementary learning interventions.

Date: 2026-07-09
References: Add references at CitEc
Citations:

Downloads: (external link)
https://osf.io/download/6a4f408ef4a26428265b92e2/

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:osf:socarx:rf8kx_v2

DOI: 10.31219/osf.io/rf8kx_v2

Access Statistics for this paper

More papers in SocArXiv from Center for Open Science
Bibliographic data for series maintained by OSF ().