Power analysis and type I and type II error rates of Bayesian nonparametric two-sample tests for location-shifts based on the Bayes factor under Cauchy priors

Riko Kelter

Computational Statistics & Data Analysis, 2022, vol. 165, issue C

Abstract: Hypothesis testing is a central statistical method in the biomedical sciences. The ongoing debate about the concept of statistical significance and the reliability of null hypothesis significance tests (NHST) and p-values has brought the advent of various Bayesian hypothesis tests as possible alternatives, which often employ the Bayes factor. However, careful calibration of the prior parameters is necessary for the type I error rates or power of these alternatives to be any better. Also, the availability of various Bayesian tests for the same statistical problem leads to the question which test to choose based on which criteria. Recently proposed Bayesian nonparametric two-sample tests are analyzed with regard to their type I error rates and power to detect an effect. Results show that approaches vary substantially in their ability to control the type I and II errors, and it is shown how to select the prior parameters to balance power and type I error control. This allows for prior elicitation and power analyses based on objective criteria like type I and II error rates even when conducting a Bayesian nonparametric two-sample test. Also, it is shown that existing nonparametric Bayesian two-sample tests are adequate only to test for location-shifts. Together, the results provide guidance how to perform a nonparametric Bayesian two-sample test while simultaneously improving the reliability of research.

Keywords: Hypothesis tests; Bayesian statistics; Nonparametric hypothesis tests; Two-sample tests; Type I error; Type II error; Reproducibility crisis (search for similar items in EconPapers)
Date: 2022
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0167947321001602
Full text for ScienceDirect subscribers only.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:csdana:v:165:y:2022:i:c:s0167947321001602

DOI: 10.1016/j.csda.2021.107326

Access Statistics for this article

Computational Statistics & Data Analysis is currently edited by S.P. Azen

More articles in Computational Statistics & Data Analysis from Elsevier
Bibliographic data for series maintained by Catherine Liu ().