Resampling Plans and the Estimation of Prediction Error

Bradley Efron
Additional contact information
Bradley Efron: Department of Statistics, Stanford University, Stanford, CA 94305, USA

Stats, 2021, vol. 4, issue 4, 1-25

Abstract: This article was prepared for the Special Issue on Resampling methods for statistical inference of the 2020s . Modern algorithms such as random forests and deep learning are automatic machines for producing prediction rules from training data. Resampling plans have been the key technology for evaluating a rule’s prediction accuracy. After a careful description of the measurement of prediction error the article discusses the advantages and disadvantages of the principal methods: cross-validation, the nonparametric bootstrap, covariance penalties (Mallows’ C p and the Akaike Information Criterion), and conformal inference. The emphasis is on a broad overview of a large subject, featuring examples, simulations, and a minimum of technical detail.

Keywords: cross-validation; Cp; AIC; Q -class; conformal inference; random forests; bagging (search for similar items in EconPapers)
JEL-codes: C1 C10 C11 C14 C15 C16 (search for similar items in EconPapers)
Date: 2021
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/2571-905X/4/4/63/pdf (application/pdf)
https://www.mdpi.com/2571-905X/4/4/63/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jstats:v:4:y:2021:i:4:p:63-1115:d:706231

Access Statistics for this article

Stats is currently edited by Mrs. Minnie Li

More articles in Stats from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().