Controlling the danger of false discoveries in estimating multiple treatment effects

Dan Wunderli

No 60, ECON - Working Papers from Department of Economics - University of Zurich

Abstract: I expose the risk of false discoveries in the context of multiple treatment effects. A false discovery is a nonexistent effect that is falsely labeled as statistically significant by its individual t-value. Labeling nonexistent effects as statistically significant has wide-ranging academic and policy-related implications, like costly false conclusions from policy evaluations. I eexamine an empirical labor market model by using state-of-the art multiple testing methods and I provide simulation evidence. By merely using individual t-values at conventional significance levels, the risk of labeling probably nonexistent treatment effects as statistically significant is unacceptably high. Individual t-values even label a number of treatment effects as significant, whereas multiple testing indicates false discoveries in these cases. Tests of a joint null hypothesis such as the well-known F-test control the risk of false discoveries only to a limited extent and do not optimally allow for rejecting individual hypotheses. Multiple testing methods control the risk of false discoveries in general while allowing for individual decisions in the sense of rejecting individual hypotheses.

Keywords: False discoveries; multiple error rates; multiple treatment effects; labor market (search for similar items in EconPapers)
JEL-codes: C12 C14 C21 C31 C41 J08 J64 (search for similar items in EconPapers)
Date: 2012-01
New Economics Papers: this item is included in nep-ecm
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.zora.uzh.ch/id/eprint/57865/1/econwp060.pdf (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:zur:econwp:060

Access Statistics for this paper

More papers in ECON - Working Papers from Department of Economics - University of Zurich Contact information at EDIRC.
Bibliographic data for series maintained by Severin Oswald ().