Using the Area Under an Estimated ROC Curve to Test the Adequacy of Binary Predictors

Robert Lieli and Yu-Chin Hsu

No 2018_1, CEU Working Papers from Department of Economics, Central European University

Abstract: We consider using the area under an empirical receiver operating characteristic (ROC) curve to test the hypothesis that a predictive index combined with a range of cutoffs performs no better than pure chance in forecasting a binary outcome. This corresponds to the null hypothesis that the area in question, denoted as AUC, is 1/2. We show that if the predictive index comes from a first stage regression model estimated over the same data set, then testing the null based on standard asymptotic normality results leads to severe size distortion in general settings. We then analytically derive the proper asymptotic null distribution of the empirical AUC in a special case; namely, when the first stage regressors are Bernoulli random variables. This distribution can be utilized to construct a fully in-sample test of H0 : AUC = 1=2 with correct size and more power than out-of-sample tests based on sample splitting, though practical application becomes cumbersome with more than two regressors.

Date: 2018-03-19
New Economics Papers: this item is included in nep-ecm
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (2)

Downloads: (external link)
https://ceu-economics-and-business.github.io/RePEc/pdf/2018_1.pdf Full text (application/pdf)

Related works:
Journal Article: Using the area under an estimated ROC curve to test the adequacy of binary predictors (2019)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:ceu:econwp:2018_1

Access Statistics for this paper

More papers in CEU Working Papers from Department of Economics, Central European University Contact information at EDIRC.
Bibliographic data for series maintained by Anita Apor ().