Oracle Inequalities for Convex Loss Functions with Non-Linear Targets
Mehmet Caner and
Anders Kock
CREATES Research Papers from Department of Economics and Business Economics, Aarhus University
Abstract:
This paper consider penalized empirical loss minimization of convex loss functions with unknown non-linear target functions. Using the elastic net penalty we establish a finite sample oracle inequality which bounds the loss of our estimator from above with high probability. If the unknown target is linear this inequality also provides an upper bound of the estimation error of the estimated parameter vector. These are new results and they generalize the econometrics and statistics literature. Next, we use the non-asymptotic results to show that the excess loss of our estimator is asymptotically of the same order as that of the oracle. If the target is linear we give sufficient conditions for consistency of the estimated parameter vector. Next, we briefly discuss how a thresholded version of our estimator can be used to perform consistent variable selection. We give two examples of loss functions covered by our framework and show how penalized nonparametric series estimation is contained as a special case and provide a finite sample upper bound on the mean square error of the elastic net series estimator.
Keywords: Empirical loss minimization; Lasso; Elastic net; Oracle inequality; Convex loss function; Nonparametric estimation; Variable selection. (search for similar items in EconPapers)
JEL-codes: C13 C21 C31 (search for similar items in EconPapers)
Pages: 44
Date: 2013-12-13
New Economics Papers: this item is included in nep-ecm
References: View references in EconPapers View complete reference list from CitEc
Citations:
Downloads: (external link)
https://repec.econ.au.dk/repec/creates/rp/13/rp13_51.pdf (application/pdf)
Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.
Export reference: BibTeX
RIS (EndNote, ProCite, RefMan)
HTML/Text
Persistent link: https://EconPapers.repec.org/RePEc:aah:create:2013-51
Access Statistics for this paper
More papers in CREATES Research Papers from Department of Economics and Business Economics, Aarhus University
Bibliographic data for series maintained by ().