A Comparative Evaluation of Multiple Hypothesis Testing Adjustment Methods

Carl Dolling

No d3svb_v1, MetaArXiv from Center for Open Science

Abstract: This paper provides empirical guidance to researchers when choosing an adjustment method during multiple hypothesis testing. It is shown that methods vary greatly in their false positive rate, not just among each other but also when the p values of the tests stem from a different distribution. It is recommended that researchers carefully choose their adjustment methods, as the choice significantly affects the interpretation of findings. When the aim of the adjustment is to control Family-Wise Error, strict and simple methods like the Bonferroni correction or Union-Intersection tests offer great practical applicability. When controlling False-Discovery Rate, more powerful methods like the Benjamini-Hochberg or Simes-Hochberg methods are more appropriate.

Date: 2026-04-04
New Economics Papers: this item is included in nep-ecm
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://osf.io/download/69d2886e73c738208e032e5c/

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:osf:metaar:d3svb_v1

DOI: 10.31219/osf.io/d3svb_v1

Access Statistics for this paper

More papers in MetaArXiv from Center for Open Science
Bibliographic data for series maintained by OSF ().