A scalable estimate of the out‐of‐sample prediction error via approximate leave‐one‐out cross‐validation

Kamiar Rahnama Rad and Arian Maleki

Journal of the Royal Statistical Society Series B, 2020, vol. 82, issue 4, 965-996

Abstract: The paper considers the problem of out‐of‐sample risk estimation under the high dimensional settings where standard techniques such as K‐fold cross‐validation suffer from large biases. Motivated by the low bias of the leave‐one‐out cross‐validation method, we propose a computationally efficient closed form approximate leave‐one‐out formula ALO for a large class of regularized estimators. Given the regularized estimate, calculating ALO requires a minor computational overhead. With minor assumptions about the data‐generating process, we obtain a finite sample upper bound for the difference between leave‐one‐out cross‐validation and approximate leave‐one‐out cross‐validation, |LO−ALO|. Our theoretical analysis illustrates that |LO−ALO|→0 with overwhelming probability, when n,p→∞, where the dimension p of the feature vectors may be comparable with or even greater than the number of observations, n. Despite the high dimensionality of the problem, our theoretical results do not require any sparsity assumption on the vector of regression coefficients. Our extensive numerical experiments show that |LO−ALO| decreases as n and p increase, revealing the excellent finite sample performance of approximate leave‐one‐out cross‐validation. We further illustrate the usefulness of our proposed out‐of‐sample risk estimation method by an example of real recordings from spatially sensitive neurons (grid cells) in the medial entorhinal cortex of a rat.

Date: 2020
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://doi.org/10.1111/rssb.12374

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:bla:jorssb:v:82:y:2020:i:4:p:965-996

Ordering information: This journal article can be ordered from
http://ordering.onli ... 1111/(ISSN)1467-9868

Access Statistics for this article

Journal of the Royal Statistical Society Series B is currently edited by P. Fryzlewicz and I. Van Keilegom

More articles in Journal of the Royal Statistical Society Series B from Royal Statistical Society Contact information at EDIRC.
Bibliographic data for series maintained by Wiley Content Delivery ().