How accurate is the accuracy ratio in credit risk model validation?

Marco van der Burgt

Journal of Risk Model Validation

Abstract: The receiver operating curve and the cumulative accuracy profile visualize the ability of a credit scoring model to distinguish defaulting from nondefaulting counterparties. These curves lead to performance metrics such as the accuracy ratio and the area under the curve. Since these performance metrics are sample properties, we cannot draw firm conclusions on the model performance without knowing the sampling distribution or the sample variance. We present four methods to estimate the sample variance of the accuracy ratio and the area under the curve. The first method is based on numerical integration, the second and third methods assume specific score distributions, and the fourth method uses a correlation, leading to a distribution-independent equation for the sample variance. We demonstrate by simulations that the first method gives the best estimation of the sample variance. The distribution-independent equation gives reasonable estimations of the sample variance, but ignores higher-order effects that are distribution dependent.

References: Add references at CitEc
Citations:

Downloads: (external link)
https://www.risk.net/journal-of-risk-model-validat ... isk-model-validation (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:rsk:journ5:7736721

Access Statistics for this article

More articles in Journal of Risk Model Validation from Journal of Risk Model Validation
Bibliographic data for series maintained by Thomas Paine ().