The degrees of freedom of partly smooth regularizers

Samuel Vaiter, Charles Deledalle, Jalal Fadili (), Gabriel Peyré and Charles Dossal
Additional contact information
Samuel Vaiter: CEREMADE, CNRS, Université Paris-Dauphine
Charles Deledalle: IMB, CNRS, Université Bordeaux 1
Jalal Fadili: Normandie Univ, ENSICAEN, CNRS, GREYC
Gabriel Peyré: CEREMADE, CNRS, Université Paris-Dauphine
Charles Dossal: IMB, CNRS, Université Bordeaux 1

Annals of the Institute of Statistical Mathematics, 2017, vol. 69, issue 4, No 4, 832 pages

Abstract: Abstract We study regularized regression problems where the regularizer is a proper, lower-semicontinuous, convex and partly smooth function relative to a Riemannian submanifold. This encompasses several popular examples including the Lasso, the group Lasso, the max and nuclear norms, as well as their composition with linear operators (e.g., total variation or fused Lasso). Our main sensitivity analysis result shows that the predictor moves locally stably along the same active submanifold as the observations undergo small perturbations. This plays a pivotal role in getting a closed-form expression for the divergence of the predictor w.r.t. observations. We also show that, for many regularizers, including polyhedral ones or the analysis group Lasso, this divergence formula holds Lebesgue a.e. When the perturbation is random (with an appropriate continuous distribution), this allows us to derive an unbiased estimator of the degrees of freedom and the prediction risk. Our results unify and go beyond those already known in the literature.

Keywords: Degrees of freedom; Partial smoothness; Manifold; Sparsity; Model selection; O-minimal structures; Semi-algebraic sets; Group Lasso; Total variation (search for similar items in EconPapers)
Date: 2017
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (3)

Downloads: (external link)
http://link.springer.com/10.1007/s10463-016-0563-z Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:aistmt:v:69:y:2017:i:4:d:10.1007_s10463-016-0563-z

Ordering information: This journal article can be ordered from
http://www.springer. ... cs/journal/10463/PS2

DOI: 10.1007/s10463-016-0563-z

Access Statistics for this article

Annals of the Institute of Statistical Mathematics is currently edited by Tomoyuki Higuchi

More articles in Annals of the Institute of Statistical Mathematics from Springer, The Institute of Statistical Mathematics
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().