Mostly Harmless Simulations? Using Monte Carlo Studies for Estimator Selection

Arun Advani, Toru Kitagawa and Tymon Słoczyński
Additional contact information
Toru Kitagawa: University College London and cemmap.

The Warwick Economics Research Paper Series (TWERPS) from University of Warwick, Department of Economics

Abstract: We consider two recent suggestions for how to perform an empirically motivated Monte Carlo study to help select a treatment effect estimator under unconfoundedness. We show theoretically that neither is likely to be informative except under restrictive conditions that are unlikely to be satisfied in many contexts. To test empirical relevance, we also apply the approaches to a real-world setting where estimator performance is known. Both approaches are worse than random at selecting estimators which minimise absolute bias. They are better when selecting estimators that minimise mean squared error. However, using a simple bootstrap is at least as good and often better. For now researchers would be best advised to use a range of estimators and compare estimates for robustness.

Keywords: Fertility; Discrimination; Experimental economics (search for similar items in EconPapers)
JEL-codes: C93 J16 J71 (search for similar items in EconPapers)
Date: 2019
New Economics Papers: this item is included in nep-exp and nep-lma
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (8)

Downloads: (external link)
https://warwick.ac.uk/fac/soc/economics/research/w ... werp_1192_advani.pdf

Related works:
Journal Article: Mostly harmless simulations? Using Monte Carlo studies for estimator selection (2019)
Working Paper: Mostly Harmless Simulations? Using Monte Carlo Studies for Estimator Selection (2019)
Working Paper: Mostly Harmless Simulations? Using Monte Carlo Studies for Estimator Selection (2019)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:wrk:warwec:1192

Access Statistics for this paper

More papers in The Warwick Economics Research Paper Series (TWERPS) from University of Warwick, Department of Economics Contact information at EDIRC.
Bibliographic data for series maintained by Margaret Nash ().