A small-sample Bayesian information criterion that does not overstate the evidence, with an application to calibrating p-values from likelihood-ratio tests

David R. Bickel ()
Additional contact information
David R. Bickel: University of North Carolina at Greensboro

Statistical Papers, 2025, vol. 66, issue 3, No 16, 17 pages

Abstract: Abstract This paper proposes a simple correction to the Bayesian information criterion (BIC) for small samples to ensure that it neither overstates nor understates the evidence against a null hypothesis or other tested model. The new correction raises the likelihood ratio in the BIC to the power of 1 minus the reciprocal of the sample size ( $$1-1/\textrm{n}, \textrm{n}>1$$ 1 - 1 / n , n > 1 ). That is equivalent to multiplying the loglikelihood term of the BIC by a factor of $$1-1/\textrm{n}.$$ 1 - 1 / n . The correction is applied to the problem of calibrating p-values by transforming them to estimated Bayes factors. The corresponding calibration in the most common case is simply sqrt(n)/exp((1−1/n)*qchisq(1−p,df=1)/2) in R syntax, where the p-value is from a likelihood-ratio test. That intersects the class of betting scores called e-values and, more specifically, admissible calibrators. While all admissible calibrators neither overstate nor understate the evidence against the null hypothesis, previous admissible calibrators are not model-selection consistent since they do not increasingly favor the null hypothesis when it is true. The proposed calibrator is consistent under general conditions, for its corrected BIC is asymptotically equivalent to the BIC.

Keywords: Corrected BIC; Corrected Bayesian information criterion; Calibrated p-value; Calibration of p-values; Exaggeration of evidence; Overstatement of evidence; Strength of statistical evidence (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
http://link.springer.com/10.1007/s00362-025-01682-1 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:stpapr:v:66:y:2025:i:3:d:10.1007_s00362-025-01682-1

Ordering information: This journal article can be ordered from
http://www.springer. ... business/journal/362

DOI: 10.1007/s00362-025-01682-1

Access Statistics for this article

Statistical Papers is currently edited by C. Müller, W. Krämer and W.G. Müller

More articles in Statistical Papers from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().