Economics at your fingertips  

Using the Area Under an Estimated ROC Curve to Test the Adequacy of Binary Predictors

Robert Lieli and Yu-Chin Hsu ()

No 2018_1, CEU Working Papers from Department of Economics, Central European University

Abstract: We consider using the area under an empirical receiver operating characteristic (ROC) curve to test the hypothesis that a predictive index combined with a range of cutoffs performs no better than pure chance in forecasting a binary outcome. This corresponds to the null hypothesis that the area in question, denoted as AUC, is 1/2. We show that if the predictive index comes from a first stage regression model estimated over the same data set, then testing the null based on standard asymptotic normality results leads to severe size distortion in general settings. We then analytically derive the proper asymptotic null distribution of the empirical AUC in a special case; namely, when the first stage regressors are Bernoulli random variables. This distribution can be utilized to construct a fully in-sample test of H0 : AUC = 1=2 with correct size and more power than out-of-sample tests based on sample splitting, though practical application becomes cumbersome with more than two regressors.

New Economics Papers: this item is included in nep-ecm
Date: 2018-03-19
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (1) Track citations by RSS feed

Downloads: (external link) Full text (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link:

Access Statistics for this paper

More papers in CEU Working Papers from Department of Economics, Central European University Contact information at EDIRC.
Bibliographic data for series maintained by Anita Apor ().

Page updated 2019-02-07
Handle: RePEc:ceu:econwp:2018_1