Using the Area Under an Estimated ROC Curve to Test the Adequacy of Binary Predictors
Robert Lieli and
No 2018_1, CEU Working Papers from Department of Economics, Central European University
We consider using the area under an empirical receiver operating characteristic (ROC) curve to test the hypothesis that a predictive index combined with a range of cutoffs performs no better than pure chance in forecasting a binary outcome. This corresponds to the null hypothesis that the area in question, denoted as AUC, is 1/2. We show that if the predictive index comes from a first stage regression model estimated over the same data set, then testing the null based on standard asymptotic normality results leads to severe size distortion in general settings. We then analytically derive the proper asymptotic null distribution of the empirical AUC in a special case; namely, when the first stage regressors are Bernoulli random variables. This distribution can be utilized to construct a fully in-sample test of H0 : AUC = 1=2 with correct size and more power than out-of-sample tests based on sample splitting, though practical application becomes cumbersome with more than two regressors.
New Economics Papers: this item is included in nep-ecm
References: Add references at CitEc
Citations Track citations by RSS feed
Downloads: (external link)
http://www.personal.ceu.hu/staff/repec/pdf/2018_1.pdf Full text (application/pdf)
This item may be available elsewhere in EconPapers: Search for items with the same title.
Export reference: BibTeX
RIS (EndNote, ProCite, RefMan)
Persistent link: https://EconPapers.repec.org/RePEc:ceu:econwp:2018_1
Access Statistics for this paper
More papers in CEU Working Papers from Department of Economics, Central European University Contact information at EDIRC.
Bibliographic data for series maintained by Anita Apor ().