Sparse kernel deep stacking networks

Thomas Welchowski () and Matthias Schmid ()
Additional contact information
Thomas Welchowski: University Hospital Bonn
Matthias Schmid: University Hospital Bonn

Computational Statistics, 2019, vol. 34, issue 3, No 4, 993-1014

Abstract: Abstract This article introduces the supervised deep learning method sparse kernel deep stacking networks (SKDSNs), which extend traditional kernel deep stacking networks (KDSNs) by incorporating a set of data-driven regularization and variable selection steps to improve predictive performance in high-dimensional settings. Before model fitting, variable pre-selection is carried out using genetic algorithms in combination with the randomized dependence coefficient, accounting for non-linear dependencies among the inputs and the outcome variable. During model fitting, internal variable selection is based on a ranked feature ordering which is tuned within the model-based optimization framework. Further regularization steps include $$L_1$$ L 1 -penalized kernel regression and dropout. Our simulation studies demonstrate an improved prediction accuracy of SKDSNs compared to traditional KDSNs. Runtime analysis of SKDSNs shows that the dimension of the random Fourier transformation greatly affects computational efficiency, and that the speed of SKDSNs can be improved by applying a subsampling-based ensemble strategy. Numerical experiments show that the latter strategy further increases predictive performance. Application of SKDSNs to three biomedical data sets confirm the results of the simulation study. SKDSNs are implemented in a new version of the R package kernDeepStackNet.

Keywords: Deep learning; Dropout; Kernel regression; Model-based optimization; Sparsity; Variable selection (search for similar items in EconPapers)
Date: 2019
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
http://link.springer.com/10.1007/s00180-018-0832-9 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:compst:v:34:y:2019:i:3:d:10.1007_s00180-018-0832-9

Ordering information: This journal article can be ordered from
http://www.springer.com/statistics/journal/180/PS2

DOI: 10.1007/s00180-018-0832-9

Access Statistics for this article

Computational Statistics is currently edited by Wataru Sakamoto, Ricardo Cao and Jürgen Symanzik

More articles in Computational Statistics from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().