A Universal Approximate Cross-Validation Criterion for Regular Risk Functions

Commenges Daniel (), Proust-Lima Cécile (), Samieri Cécilia () and Liquet Benoit ()
Additional contact information
Commenges Daniel: INSERM, ISPED, Centre INSERM U-897-Epidemiologie-Biostatistique, Bordeaux F-33000, France ISPED, Centre INSERM U-897-Epidemiologie-Biostatistique, Universite de Bordeaux, Bordeaux F-33000, France
Proust-Lima Cécile: INSERM, ISPED, Centre INSERM U-897-Epidemiologie-Biostatistique, Bordeaux F-33000, France ISPED, Centre INSERM U-897-Epidemiologie-Biostatistique, Universite de Bordeaux, Bordeaux F-33000, France
Samieri Cécilia: INSERM, ISPED, Centre INSERM U-897-Epidemiologie-Biostatistique, Bordeaux F-33000, France ISPED, Centre INSERM U-897-Epidemiologie-Biostatistique, Universite de Bordeaux, Bordeaux F-33000, France
Liquet Benoit: INSERM, ISPED, Centre INSERM U-897-Epidemiologie-Biostatistique, Bordeaux F-33000, France ISPED, Centre INSERM U-897-Epidemiologie-Biostatistique, Universite de Bordeaux, Bordeaux F-33000, France School of Mathematics and Physics, The University of Queensland, St Lucia, Brisbane, Queensland 4066, Australia

The International Journal of Biostatistics, 2015, vol. 11, issue 1, 51-67

Abstract: Selection of estimators is an essential task in modeling. A general framework is that the estimators of a distribution are obtained by minimizing a function (the estimating function) and assessed using another function (the assessment function). A classical case is that both functions estimate an information risk (specifically cross-entropy); this corresponds to using maximum likelihood estimators and assessing them by Akaike information criterion (AIC). In more general cases, the assessment risk can be estimated by leave-one-out cross-validation. Since leave-one-out cross-validation is computationally very demanding, we propose in this paper a universal approximate cross-validation criterion under regularity conditions (UACVR). This criterion can be adapted to different types of estimators, including penalized likelihood and maximum a posteriori estimators, and also to different assessment risk functions, including information risk functions and continuous rank probability score (CRPS). UACVR reduces to Takeuchi information criterion (TIC) when cross-entropy is the risk for both estimation and assessment. We provide the asymptotic distributions of UACVR and of a difference of UACVR values for two estimators. We validate UACVR using simulations and provide an illustration on real data both in the psychometric context where estimators of the distributions of ordered categorical data derived from threshold models and models based on continuous approximations are compared.

Keywords: AIC; cross-entropy; cross-validation; estimator choice; Kullback–Leibler risk; model selection; ordered categorical observations; psychometric tests (search for similar items in EconPapers)
Date: 2015
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://doi.org/10.1515/ijb-2015-0004 (text/html)
For access to full text, subscription to the journal or payment for the individual article is required.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:bpj:ijbist:v:11:y:2015:i:1:p:51-67:n:10

Ordering information: This journal article can be ordered from
https://www.degruyter.com/journal/key/ijb/html

DOI: 10.1515/ijb-2015-0004

Access Statistics for this article

The International Journal of Biostatistics is currently edited by Antoine Chambaz, Alan E. Hubbard and Mark J. van der Laan

More articles in The International Journal of Biostatistics from De Gruyter
Bibliographic data for series maintained by Peter Golla ().