A Simple Measure of Robustness for External Validity under Covariate Shifts

Pietro Emilio Spini

Papers from arXiv.org

Abstract: This paper studies the robustness of estimated policy effects to changes in the distribution of covariates, a key determinant of the external validity of (quasi)-experimental results. I propose a novel robustness metric $\delta^*$ which measures the smallest covariate shift needed to invalidate an empirical claim about the policy effect (e.g., $ATE > 0$). I estimate $\delta^*$ via de-biased GMM, achieving a parametric rate of convergence while accommodating machine-learning estimators of treatment-effect heterogeneity (e.g., LASSO, random forests, neural networks). I develop benchmarking and calibration exercises to interpret the magnitude of $\delta^*$. I illustrate these tools in an application to the Oregon Health Insurance Experiment. Researchers can report $\delta^*$ alongside the point estimate and standard error as a third number gauging external validity under covariate shifts.

Date: 2021-12, Revised 2026-05
New Economics Papers: this item is included in nep-big, nep-cmp, nep-ecm and nep-ias
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (5)

Downloads: (external link)
http://arxiv.org/pdf/2112.09259 Latest version (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:arx:papers:2112.09259

Access Statistics for this paper

More papers in Papers from arXiv.org
Bibliographic data for series maintained by arXiv administrators ().