The Null Distribution of the Empirical AUC for Classi ers with Estimated Parameters: a Special Case

Robert Lieli and Yu-Chin Hsu

No 16-A007, IEAS Working Paper : academic research from Institute of Economics, Academia Sinica, Taipei, Taiwan

Abstract: We study the distribution of the area under an empirical receiver operating characteristic (ROC) curve constructed from a first stage regression model with parameters estimated on the same data set. We provide a general, but somewhat intrinsic, characterization of the limit distribution of this area, denoted AUC, when the regressors are Bernoulli random variables jointly independent of the outcome. Using the general theory, we further analyze the limit distribution in the two regressor case. It is non-normal and right-skewed. Though the theory applies, explicit expressions for the limit distribution are cumbersome to write down for a larger number of regressors. We provide a trivariate example as further illustration.

Keywords: binary classification; ROC curve; area under the ROC curve; overfitting; hypothesis testing; model selection (search for similar items in EconPapers)
Pages: 21 pages
Date: 2016-06
New Economics Papers: this item is included in nep-ecm
References: Add references at CitEc
Citations:

Downloads: (external link)
https://www.econ.sinica.edu.tw/~econ/pdfPaper/16-A007.pdf (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:sin:wpaper:16-a007

Access Statistics for this paper

More papers in IEAS Working Paper : academic research from Institute of Economics, Academia Sinica, Taipei, Taiwan Contact information at EDIRC.
Bibliographic data for series maintained by HsiaoyunLiu ().