Sparse Besov Space Analysis of Representations in Machine Learning

Ido Ben Shaul () and Shai Dekel ()
Additional contact information
Ido Ben Shaul: Tel Aviv University, School of Mathematical Sciences
Shai Dekel: Tel Aviv University, School of Mathematical Sciences

A chapter in Multiscale, Nonlinear and Adaptive Approximation II, 2024, pp 75-97 from Springer

Abstract: Abstract We present a survey of a unified approach that analyzes representation spaces in signal processing, classic machine learning and deep learning using familiar concepts of sparsity and smoothness spaces. The theory is especially suited for high dimensional spaces. It is known that it is not possible to characterize the approximation spaces of deep learning models using classic smoothness spaces [19]. Furthermore, many problems solved by deep learning are high dimensional where classical function spaces such as the isotropic Besov spaces are somewhat inadequate. Here, we shed some light on this problem by analyzing the dynamics of sparse Besov function smoothness of representations across the layers of a deep neural network, during and after training. We justify our approach by extensive experiments demonstrating that in well-performing trained networks, the sparse Besov smoothness of the training set, measured in the corresponding hidden layer feature map representation, increases from layer to layer.

Date: 2024
References: Add references at CitEc
Citations:

There are no downloads for this item, see the EconPapers FAQ for hints about obtaining it.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:sprchp:978-3-031-75802-7_5

Ordering information: This item can be ordered from
http://www.springer.com/9783031758027

DOI: 10.1007/978-3-031-75802-7_5

Access Statistics for this chapter

More chapters in Springer Books from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().