The effect of variant sample sizes and default rates on validation metrics for probability of default models

David Li, Ruchi Bhariok and Radu Neagu

Journal of Risk Model Validation

Abstract: ABSTRACT In this paper a survey of common model-validation metrics for scoring models where the response is a binary variable is presented. These metrics include the Hosmer-Lemeshow statistic, the accuracy ratio, the standardized residual sum of squares and the conditional information entropy ratio. More specifically, we restrict ourselves to probability of obligor credit default models, and investigate the effects of varying sample sizes and default rates in the population. We show that no single validation metric gives accurate evaluations for a set of varying conditions, and we document the weaknesses and the strengths of these metrics using simulation and empirical data. We recommend that decision makers use information from multiple sources to drive their decisions, and that they understand the weight they need to put on each source given the specifics of the situation at hand.

References: Add references at CitEc
Citations:

Downloads: (external link)
https://www.risk.net/journal-of-risk-model-validat ... ty-of-default-models (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:rsk:journ5:2161287

Access Statistics for this article

More articles in Journal of Risk Model Validation from Journal of Risk Model Validation
Bibliographic data for series maintained by Thomas Paine ().