Using the area under an estimated ROC curve to test the adequacy of binary predictors

Robert Lieli and Yu-Chin Hsu

Journal of Nonparametric Statistics, 2019, vol. 31, issue 1, 100-130

Abstract: We consider using the area under an empirical receiver operating characteristic curve to test the hypothesis that a predictive index combined with a range of cutoffs performs no better than pure chance in forecasting a binary outcome. This corresponds to the null hypothesis that the area in question, denoted as AUC, is 1/2. We show that if the predictive index comes from a first-stage regression model estimated over the same data set, then testing the null based on the standard asymptotic normality results leads to severe size distortion in general settings. We then analytically derive the proper asymptotic null distribution of the empirical AUC in a special case; namely, when the first-stage regressors are Bernoulli random variables. This distribution can be utilised to construct a fully in-sample test of $ H_0: {\rm AUC}=1/2 $ H0:AUC=1/2 with correct size and more power than out-of-sample tests based on sample splitting, though practical application becomes cumbersome with more than two regressors.

Date: 2019
References: Add references at CitEc
Citations: View citations in EconPapers (7)

Downloads: (external link)
http://hdl.handle.net/10.1080/10485252.2018.1537440 (text/html)
Access to full text is restricted to subscribers.

Related works:
Working Paper: Using the Area Under an Estimated ROC Curve to Test the Adequacy of Binary Predictors (2018)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:taf:gnstxx:v:31:y:2019:i:1:p:100-130

Ordering information: This journal article can be ordered from
http://www.tandfonline.com/pricing/journal/GNST20

DOI: 10.1080/10485252.2018.1537440

Access Statistics for this article

Journal of Nonparametric Statistics is currently edited by Jun Shao

More articles in Journal of Nonparametric Statistics from Taylor & Francis Journals
Bibliographic data for series maintained by Chris Longhurst ().