Mostly Harmless Simulations? On the Internal Validity of Empirical Monte Carlo Studies

Arun Advani and Tymon Słoczyński

No 7874, IZA Discussion Papers from Institute of Labor Economics (IZA)

Abstract: In this paper we evaluate the premise from the recent literature on Monte Carlo studies that an empirically motivated simulation exercise is informative about the actual ranking of various estimators when applied to a particular problem. We consider two alternative designs and provide an empirical test for both of them. We conclude that a necessary condition for the simulations to be informative about the true ranking is that the treatment effect in simulations must be equal to the (unknown) true effect. This severely limits the usefulness of such procedures, since were the effect known, the procedure would not be necessary.

Keywords: empirical Monte Carlo studies; programme evaluation; treatment effects (search for similar items in EconPapers)
JEL-codes: C15 C21 C25 C52 (search for similar items in EconPapers)
Pages: 35 pages
Date: 2013-12
New Economics Papers: this item is included in nep-ecm and nep-ore
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (4)

Forthcoming - superseded by IZA Discussion Paper No. 11862

Downloads: (external link)
https://docs.iza.org/dp7874.pdf (application/pdf)

Related works:
Working Paper: Mostly Harmless Simulations? On the Internal Validity of Empirical Monte Carlo Studies (2018)
Working Paper: Mostly harmless simulations? On the internal validity of empirical Monte Carlo studies (2018)
Working Paper: Mostly Harmless Simulations? On the Internal Validity of Empirical Monte Carlo Studies (2018)
Working Paper: Mostly harmless simulations? On the internal validity of empirical Monte Carlo studies (2013)
Working Paper: Mostly harmless simulations? On the internal validity of empirical Monte Carlo studies (2013)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:iza:izadps:dp7874

Ordering information: This working paper can be ordered from
IZA, Margard Ody, P.O. Box 7240, D-53072 Bonn, Germany

Access Statistics for this paper

More papers in IZA Discussion Papers from Institute of Labor Economics (IZA) IZA, P.O. Box 7240, D-53072 Bonn, Germany. Contact information at EDIRC.
Bibliographic data for series maintained by Holger Hinte ().