A kernel‐expanded stochastic neural network

Yan Sun and Faming Liang

Journal of the Royal Statistical Society Series B, 2022, vol. 84, issue 2, 547-578

Abstract: The deep neural network suffers from many fundamental issues in machine learning. For example, it often gets trapped into a local minimum in training, and its prediction uncertainty is hard to be assessed. To address these issues, we propose the so‐called kernel‐expanded stochastic neural network (K‐StoNet) model, which incorporates support vector regression as the first hidden layer and reformulates the neural network as a latent variable model. The former maps the input vector into an infinite dimensional feature space via a radial basis function kernel, ensuring the absence of local minima on its training loss surface. The latter breaks the high‐dimensional non‐convex neural network training problem into a series of low‐dimensional convex optimization problems, and enables its prediction uncertainty easily assessed. The K‐StoNet can be easily trained using the imputation‐regularized optimization algorithm. Compared to traditional deep neural networks, K‐StoNet possesses a theoretical guarantee to asymptotically converge to the global optimum and enables the prediction uncertainty easily assessed. The performances of the new model in training, prediction and uncertainty quantification are illustrated by simulated and real data examples.

Date: 2022
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://doi.org/10.1111/rssb.12496

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:bla:jorssb:v:84:y:2022:i:2:p:547-578

Ordering information: This journal article can be ordered from
http://ordering.onli ... 1111/(ISSN)1467-9868

Access Statistics for this article

Journal of the Royal Statistical Society Series B is currently edited by P. Fryzlewicz and I. Van Keilegom

More articles in Journal of the Royal Statistical Society Series B from Royal Statistical Society Contact information at EDIRC.
Bibliographic data for series maintained by Wiley Content Delivery ().