On the effectiveness of partially deterministic Bayesian neural networks

Daniel Andrade () and Koki Sato
Additional contact information
Daniel Andrade: Hiroshima University
Koki Sato: NTT Data MHI Systems

Computational Statistics, 2025, vol. 40, issue 5, No 8, 2518 pages

Abstract: Abstract Bayesian neural networks (BNNs) with computationally expensive Hamiltonian Monte Carlo sampling methods are often considered to provide better predictive performance than the maximum a posterior (MAP) solution. Here, as an alternative to sampling all parameters of a BNN (full-random), we experimentally evaluate partially deterministic BNNs that fix some part of the neural network parameters to their MAP solution. In particular, we consider various strategies for fixing half, or all parameters of a layer to the MAP-solution. Over a wide variety of regression and classification tasks, we find that partially deterministic BNNs often significantly improve predictive performance over the MAP-solution, with up to around 24% reduction in negative log-likelihood. Notably, we also find that partially deterministic BNNs that fix half of the parameters in each layer can also reduce under-fitting of full-random BNNs, resulting in up to 7% reduction in negative log-likelihood.

Keywords: Stochastic gradient Hamiltonian Monte Carlo; Computational efficiency; Multilayer perceptron; uncertainty quantification; Out-of-distribution data (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
http://link.springer.com/10.1007/s00180-024-01561-7 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:compst:v:40:y:2025:i:5:d:10.1007_s00180-024-01561-7

Ordering information: This journal article can be ordered from
http://www.springer.com/statistics/journal/180/PS2

DOI: 10.1007/s00180-024-01561-7

Access Statistics for this article

Computational Statistics is currently edited by Wataru Sakamoto, Ricardo Cao and Jürgen Symanzik

More articles in Computational Statistics from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().