Bernstein flows for flexible posteriors in variational Bayes

Oliver Dürr (), Stefan Hörtling (), Danil Dold (), Ivonne Kovylov () and Beate Sick ()
Additional contact information
Oliver Dürr: Konstanz University of Applied Sciences
Stefan Hörtling: Konstanz University of Applied Sciences
Danil Dold: Konstanz University of Applied Sciences
Ivonne Kovylov: Konstanz University of Applied Sciences
Beate Sick: Zurich University of Applied Sciences

AStA Advances in Statistical Analysis, 2024, vol. 108, issue 2, No 7, 375-394

Abstract: Abstract Black-box variational inference (BBVI) is a technique to approximate the posterior of Bayesian models by optimization. Similar to MCMC, the user only needs to specify the model; then, the inference procedure is done automatically. In contrast to MCMC, BBVI scales to many observations, is faster for some applications, and can take advantage of highly optimized deep learning frameworks since it can be formulated as a minimization task. In the case of complex posteriors, however, other state-of-the-art BBVI approaches often yield unsatisfactory posterior approximations. This paper presents Bernstein flow variational inference (BF-VI), a robust and easy-to-use method flexible enough to approximate complex multivariate posteriors. BF-VI combines ideas from normalizing flows and Bernstein polynomial-based transformation models. In benchmark experiments, we compare BF-VI solutions with exact posteriors, MCMC solutions, and state-of-the-art BBVI methods, including normalizing flow-based BBVI. We show for low-dimensional models that BF-VI accurately approximates the true posterior; in higher-dimensional models, BF-VI compares favorably against other BBVI methods. Further, using BF-VI, we develop a Bayesian model for the semi-structured melanoma challenge data, combining a CNN model part for image data with an interpretable model part for tabular data, and demonstrate, for the first time, the use of BBVI in semi-structured models.

Keywords: Variational inference; Deep learning; Transformation models; Bayesian neural network (search for similar items in EconPapers)
Date: 2024
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
http://link.springer.com/10.1007/s10182-024-00497-z Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:alstar:v:108:y:2024:i:2:d:10.1007_s10182-024-00497-z

Ordering information: This journal article can be ordered from
http://www.springer. ... cs/journal/10182/PS2

DOI: 10.1007/s10182-024-00497-z

Access Statistics for this article

AStA Advances in Statistical Analysis is currently edited by Göran Kauermann and Yarema Okhrin

More articles in AStA Advances in Statistical Analysis from Springer, German Statistical Society
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().