A comment on “A Systematic Review and Meta-analysis of the Evidence on Learning During the COVID-19 Pandemicâ€

Nino Buliskeria, Ali Elminejad, Tomas Havranek, Zuzana Irsova, Stepan Jurajda, Marek Kapicka and Martina Lušková

No 2024/41, Working Papers IES from Charles University Prague, Faculty of Social Sciences, Institute of Economic Studies

Abstract: Betthäuser et al. (2023) examine the effects of the COVID-19 pandemic on the learning progress of school-aged children. They collect 291 estimates from 42 studies. Their meta-analysis-corrected estimate implies a substantial decline in students’ learning (Cohen’s d = −0.14, 95% confidence interval −0.17 to −0.10). First, we successfully reproduce the main results and the majority of supporting figures. Second, we provide additional analysis addressing publication bias by implementing correction techniques: PET-PEESE (funnelbased), 3PSM (selection model), and RoBMA (model averaging). Additionally, we implement novel approaches that account for the strength of biased selection favoring affirmative results in the sample of analyzed studies. Third, we use techniques that assume the presence of p-hacking (MAIVE, RTMA). Using these methods, the corrected effect ranges from −0.25 to −0.11 with high statistical significance. While our analysis does reveal some evidence of publication bias and p-hacking, these phenomena do not appear to systematically distort the overall findings of the original study.

Keywords: Replication; Robustness; Meta-analysis; COVID-19; Education; Learning deficit (search for similar items in EconPapers)
JEL-codes: C68 I21 I24 I28 (search for similar items in EconPapers)
Pages: 38 pages
Date: 2024-41, Revised 2024
References: Add references at CitEc
Citations:

Downloads: (external link)
https://ies.fsv.cuni.cz/en/comment-systematic-revi ... ng-covid-19-pandemic (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:fau:wpaper:wp2024_41

Access Statistics for this paper

More papers in Working Papers IES from Charles University Prague, Faculty of Social Sciences, Institute of Economic Studies Contact information at EDIRC.
Bibliographic data for series maintained by Natalie Svarcova ().