Gradient-based smoothing parameter estimation for neural P-splines

Lea M. Dammann, Marei Freitag (), Anton Thielmann and Benjamin Säfken
Additional contact information
Lea M. Dammann: University of Göttingen
Marei Freitag: University of Göttingen
Anton Thielmann: Clausthal University of Technology
Benjamin Säfken: Clausthal University of Technology

Computational Statistics, 2025, vol. 40, issue 7, No 11, 3645-3663

Abstract: Abstract Due to the popularity of deep learning models there have recently been many attempts to translate generalized additive models to neural nets. Generalized additive models are usually regularized by a penalty in the loss function and the magnitude of penalization is controlled by one or more smoothing parameters. In the statistical literature these smoothing parameters are estimated by criteria such as generalized cross-validation or restricted maximum likelihood. While the estimation of the primary regression coefficients is well calibrated and investigated for neural net based additive models, the estimation of smoothing parameters is often either based on testing data (and grid search), implicitly estimated or completely neglected. In this paper, we address the issue of explicit smoothing parameter estimation in neural net-based additive models fitted via gradient-based methods, such as the well-known Adam algorithm. We therefore investigate the data-driven smoothing parameter selection via gradient-based optimization of generalized cross-validation and restricted maximum likelihood. Thus we do not need to calculate Hessian information of the smoothing parameters. As an additive model structure, we use a translation of P-splines to neural nets, so-called neural P-splines. The fitting process of neural P-splines as well as the gradient-based smoothing parameter selection are investigated in a simulation study and an application.

Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
http://link.springer.com/10.1007/s00180-024-01593-z Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:compst:v:40:y:2025:i:7:d:10.1007_s00180-024-01593-z

Ordering information: This journal article can be ordered from
http://www.springer.com/statistics/journal/180/PS2

DOI: 10.1007/s00180-024-01593-z

Access Statistics for this article

Computational Statistics is currently edited by Wataru Sakamoto, Ricardo Cao and Jürgen Symanzik

More articles in Computational Statistics from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().