The cross-validated adaptive epsilon-net estimator

Laan Mark J. van der, Dudoit Sandrine and Vaart Aad W. van der

Statistics & Risk Modeling, 2006, vol. 24, issue 3, 373-395

Abstract: Suppose that we observe a sample of independent and identically distributed realizations of a random variable, and a parameter of interest can be defined as the minimizer, over a suitably defined parameter set, of the expectation of a (loss) function of a candidate parameter value and the random variable. For example, squared error loss in regression or the negative log-density loss in density estimation. Minimizing the empirical risk (i.e., the empirical mean of the loss function) over the entire parameter set may result in ill-defined or too variable estimators of the parameter of interest. In this article, we propose a cross-validated ε-net estimation method, which uses a collection of submodels and a collection of ε-nets over each submodel. For each submodel s and each resolution level ε, the minimizer of the empirical risk over the corresponding ε-net is a candidate estimator. Next we select from these estimators (i.e. select the pair (s,ε)) by multi-fold cross-validation. We derive a finite sample inequality that shows that the resulting estimator is as good as an oracle estimator that uses the best submodel and resolution level for the unknown true parameter. We also address the implementation of the estimation procedure, and in the context of a linear regression model we present results of a preliminary simulation study comparing the cross-validated ε-net estimator to the cross-validated L1-penalized least squares estimator (LASSO) and the least angle regression estimator (LARS).

Keywords: adaptation; covering number; cross-validation; loss function; maximum likelihood estimation (search for similar items in EconPapers)
Date: 2006
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (10)

Downloads: (external link)
https://doi.org/10.1524/stnd.2006.24.3.373 (text/html)
For access to full text, subscription to the journal or payment for the individual article is required.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:bpj:strimo:v:24:y:2006:i:3:p:23:n:4

Ordering information: This journal article can be ordered from
https://www.degruyter.com/journal/key/strm/html

DOI: 10.1524/stnd.2006.24.3.373

Access Statistics for this article

Statistics & Risk Modeling is currently edited by Robert Stelzer

More articles in Statistics & Risk Modeling from De Gruyter
Bibliographic data for series maintained by Peter Golla ().