Statistical guarantees for sparse deep learning

Johannes Lederer ()
Additional contact information
Johannes Lederer: Ruhr-University Bochum

AStA Advances in Statistical Analysis, 2024, vol. 108, issue 2, No 2, 258 pages

Abstract: Abstract Neural networks are becoming increasingly popular in applications, but our mathematical understanding of their potential and limitations is still limited. In this paper, we further this understanding by developing statistical guarantees for sparse deep learning. In contrast to previous work, we consider different types of sparsity, such as few active connections, few active nodes, and other norm-based types of sparsity. Moreover, our theories cover important aspects that previous theories have neglected, such as multiple outputs, regularization, and $$\ell_{2}$$ ℓ 2 -loss. The guarantees have a mild dependence on network widths and depths, which means that they support the application of sparse but wide and deep networks from a statistical perspective. Some of the concepts and tools that we use in our derivations are uncommon in deep learning and, hence, might be of additional interest.

Keywords: Sparsity; Regularization; Oracle inequalities; High-dimensionality (search for similar items in EconPapers)
Date: 2024
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
http://link.springer.com/10.1007/s10182-022-00467-3 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:alstar:v:108:y:2024:i:2:d:10.1007_s10182-022-00467-3

Ordering information: This journal article can be ordered from
http://www.springer. ... cs/journal/10182/PS2

DOI: 10.1007/s10182-022-00467-3

Access Statistics for this article

AStA Advances in Statistical Analysis is currently edited by Göran Kauermann and Yarema Okhrin

More articles in AStA Advances in Statistical Analysis from Springer, German Statistical Society
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().