 Null Hypothesis Significance Testing Defended and Calibrated by Bayesian Model CheckingDavid R. Bickel The American Statistician, 2021, vol. 75, issue 3, 249-255 Abstract: Significance testing is often criticized because p-values can be low even though posterior probabilities of the null hypothesis are not low according to some Bayesian models. Those models, however, would assign low prior probabilities to the observation that the p-value is sufficiently low. That conflict between the models and the data may indicate that the models needs revision. Indeed, if the p-value is sufficiently small while the posterior probability according to a model is insufficiently small, then the model will fail a model check. That result leads to a way to calibrate a p-value by transforming it into an upper bound on the posterior probability of the null hypothesis (conditional on rejection) for any model that would pass the check. The calibration may be calculated from a prior probability of the null hypothesis and the stringency of the check without more detailed modeling. An upper bound, as opposed to a lower bound, can justify concluding that the null hypothesis has a low posterior probability. Date: 2021 Downloads: (external link) Related works: Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text Persistent link: https://EconPapers.repec.org/RePEc:taf:amstat:v:75:y:2021:i:3:p:249-255 Ordering information: This journal article can be ordered from DOI: 10.1080/00031305.2019.1699443 Access Statistics for this article The American Statistician is currently edited by Eric Sampson More articles in The American Statistician from Taylor & Francis Journals |
Is your work missing from RePEc? Here is how to contribute.
Questions or problems? Check the EconPapers FAQ or send mail to .
EconPapers is hosted by the
School of Business at Örebro University.