Reusing Natural Experiments
Davidson Heath (),
Mehrdad Samadi () and
Ingrid M. Werner
Additional contact information
Mehrdad Samadi: https://www.federalreserve.gov/econres/mehrdad-samadi.htm
Ingrid M. Werner: https://fisher.osu.edu/people/werner.47
No 1339, International Finance Discussion Papers from Board of Governors of the Federal Reserve System (U.S.)
After a natural experiment is first used, other researchers often reuse the setting, examining different outcome variables. We use simulations based on real data to illustrate the multiple hypothesis testing problem that arises when researchers reuse natural experiments. We then provide guidance for future inference based on popular empirical settings including difference-in-differences regressions, instrumental variables regressions, and regression discontinuity designs. When we apply our guidance to two extensively studied natural experiments, business combination laws and the Regulation SHO pilot, we find that many results that were statistically significant using single hypothesis testing do not survive corrections for multiple hypothesis testing.
Keywords: False Positive; Multiple Hypothesis Testing; Natural Experiments (search for similar items in EconPapers)
JEL-codes: G10 (search for similar items in EconPapers)
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (1) Track citations by RSS feed
Downloads: (external link)
Working Paper: Reusing Natural Experiments (2020)
Working Paper: Reusing Natural Experiments (2019)
This item may be available elsewhere in EconPapers: Search for items with the same title.
Export reference: BibTeX
RIS (EndNote, ProCite, RefMan)
Persistent link: https://EconPapers.repec.org/RePEc:fip:fedgif:1339
Access Statistics for this paper
More papers in International Finance Discussion Papers from Board of Governors of the Federal Reserve System (U.S.) Contact information at EDIRC.
Bibliographic data for series maintained by Ryan Wolfslayer ; Keisha Fournillier ().