A Comment on "A Systematic Review and Meta-Analysis of the Evidence on Learning During the COVID-19 Pandemic"

Nino Buliskeria, Ali Elminejad, Tomas Havranek, Zuzana Irsova, Stepan Jurajda, Marek Kapicka and Martina Lušková

No 223, I4R Discussion Paper Series from The Institute for Replication (I4R)

Abstract: Betthäuser et al. (2023) examine the effects of the COVID-19 pandemic on the learning progress of school-aged children. They collect 291 estimates from 42 studies. Their meta-analysis-corrected estimate implies a substantial decline in students' learning (Cohen's d = −0.14, 95% confidence interval −0.17 to −0.10). First, we successfully reproduce the main results and the majority of supporting figures. Second, we provide additional analysis addressing publication bias by implementing correction techniques: PET-PEESE (funnelbased), 3PSM (selection model), and RoBMA (model averaging). Additionally, we implement novel approaches that account for the strength of biased selection favoring affirmative results in the sample of analyzed studies. Third, we use techniques that assume the presence of p-hacking (MAIVE, RTMA). Using these methods, the corrected effect ranges from −0.25 to −0.11 with high statistical significance. While our analysis does reveal some evidence of selection bias in underlying data (primary studies), these phenomena do not appear to systematically distort the overall findings of the original study.

Keywords: Replication; Robustness; Meta-analysis; COVID-19; Education; Learning deficit (search for similar items in EconPapers)
JEL-codes: C68 I21 I24 I28 (search for similar items in EconPapers)
Date: 2025
New Economics Papers: this item is included in nep-hea
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.econstor.eu/bitstream/10419/315577/1/I4R-DP223.pdf (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:zbw:i4rdps:223

Access Statistics for this paper

More papers in I4R Discussion Paper Series from The Institute for Replication (I4R)
Bibliographic data for series maintained by ZBW - Leibniz Information Centre for Economics ().