General Bayesian loss function selection and the use of improper models

Jack Jewson and David Rossell

Journal of the Royal Statistical Society Series B, 2022, vol. 84, issue 5, 1640-1665

Abstract: Statisticians often face the choice between using probability models or a paradigm defined by minimising a loss function. Both approaches are useful and, if the loss can be re‐cast into a proper probability model, there are many tools to decide which model or loss is more appropriate for the observed data, in the sense of explaining the data's nature. However, when the loss leads to an improper model, there are no principled ways to guide this choice. We address this task by combining the Hyvärinen score, which naturally targets infinitesimal relative probabilities, and general Bayesian updating, which provides a unifying framework for inference on losses and models. Specifically we propose the ℋ$$ \mathscr{H} $$‐score, a general Bayesian selection criterion and prove that it consistently selects the (possibly improper) model closest to the data‐generating truth in Fisher's divergence. We also prove that an associated ℋ$$ \mathscr{H} $$‐posterior consistently learns optimal hyper‐parameters featuring in loss functions, including a challenging tempering parameter in generalised Bayesian inference. As salient examples, we consider robust regression and non‐parametric density estimation where popular loss functions define improper models for the data and hence cannot be dealt with using standard model selection tools. These examples illustrate advantages in robustness‐efficiency trade‐offs and enable Bayesian inference for kernel density estimation, opening a new avenue for Bayesian non‐parametrics.

Date: 2022
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (2)

Downloads: (external link)
https://doi.org/10.1111/rssb.12553

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:bla:jorssb:v:84:y:2022:i:5:p:1640-1665

Ordering information: This journal article can be ordered from
http://ordering.onli ... 1111/(ISSN)1467-9868

Access Statistics for this article

Journal of the Royal Statistical Society Series B is currently edited by P. Fryzlewicz and I. Van Keilegom

More articles in Journal of the Royal Statistical Society Series B from Royal Statistical Society Contact information at EDIRC.
Bibliographic data for series maintained by Wiley Content Delivery ().