A comment on "Pandemic Schooling Mode and Student Test Scores: Evidence from US School Districts"

Shumi Akhtar, Liwen Guo, Yue Hua and Hang Anh Nguyen

No 247, I4R Discussion Paper Series from The Institute for Replication (I4R)

Abstract: Jack et al. (2023) estimated the impact of district-level schooling modes (in-person, hybrid, or virtual learning) during the 2020-2021 academic year on standardized test pass rates for grades 3-8 across 11 U.S. states. Between 2019 and 2021, average pass rates declined by 12.8 percentage points in mathematics and 6.8 percentage points in English Language Arts (ELA). By leveraging within-state and commuting zone variations, the study found that districts with full in-person learning experienced significantly smaller declines-13.4 percentage points in math and 8.3 percentage points in ELA. Furthermore, the benefits of in-person learning were particularly pronounced in districts with higher proportions of Black students. The study highlights the potential long-term effects of pandemic-related schooling disruptions and highlights the need for targeted policy interventions to address learning losses. In this report, we computationally reproduced all the main results from Jack et al. (2023) using the replication package provided, including code and data. Our replication confirmed that the estimates, directions, and significance levels were identical to those reported in the published study. Additionally, we conducted two robustness tests: (1) comparing balanced versus unbalanced panel datasets and (2) examining the influence of the distribution of the independent variable (e.g., the share of in-person versus online or hybrid learning) on student performance. Our analysis shows some evidence of outliers and state-level heterogeneity in the distribution of in-person learning, which led to some deviations from the original results in the coefficient magnitudes, but the direction and significance remain similar to original results.

Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
https://www.econstor.eu/bitstream/10419/321374/1/I4R-DP247.pdf (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:zbw:i4rdps:247

Access Statistics for this paper

More papers in I4R Discussion Paper Series from The Institute for Replication (I4R)
Bibliographic data for series maintained by ZBW - Leibniz Information Centre for Economics ().