Estimation bounds and sharp oracle inequalities of regularized procedures with Lipschitz loss functions

Pierre Alquier (), Vincent Cottet () and Guillaume Lecué ()
Additional contact information
Pierre Alquier: CREST; ENSAE; Université Paris Saclay
Vincent Cottet: CREST; ENSAE; Université Paris Saclay
Guillaume Lecué: CREST; CNRS; Université Paris Saclay

No 2017-30, Working Papers from Center for Research in Economics and Statistics

Abstract: We obtain estimation error rates and sharp oracle inequalities for regularization procedures of the form [See the abstract on the paper for the formula] when ||.|| is any norm, F is a convex class of functions and l is a Lipschitz loss function satisfying a Bernstein condition over F. We explore both the bounded and subgaussian stochastic frameworks for the distribution of the f(Xi)'s, with no assumption on the distribution of the Yi's. The general results rely on two main objects: a complexity function, and a sparsity equation, that depend on the specific setting in hand (loss l and norm ||.||). As a proof of concept, we obtain minimax rates of convergence in the following problems: 1) matrix completion with any Lipschitz loss function, including the hinge and logistic loss for the so-called 1-bit matrix completion instance of the problem, and quantile losses for the general case, which enables to estimate any quantile on the entries of the matrix; 2) logistic LASSO and variants such as the logistic SLOPE; 3) kernel methods, where the loss is the hinge loss, and the regularization function is the RKHS norm.

Pages: 43 pages
Date: 2017-02-01
New Economics Papers: this item is included in nep-upt
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://crest.science/RePEc/wpstorage/2017-30.pdf CREST working paper version (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:crs:wpaper:2017-30

Access Statistics for this paper

More papers in Working Papers from Center for Research in Economics and Statistics Contact information at EDIRC.
Bibliographic data for series maintained by Secretariat General () and Murielle Jules Maintainer-Email : murielle.jules@ensae.Fr.