Measuring the model risk-adjusted performance of machine learning algorithms in credit default prediction

Andres Alonso and José Manuel Carbó Martínez ()
Additional contact information
José Manuel Carbó Martínez: Banco de España

Financial Innovation, 2022, vol. 8, issue 1, 1-35

Abstract: Abstract Implementing new machine learning (ML) algorithms for credit default prediction is associated with better predictive performance; however, it also generates new model risks, particularly concerning the supervisory validation process. Recent industry surveys often mention that uncertainty about how supervisors might assess these risks could be a barrier to innovation. In this study, we propose a new framework to quantify model risk-adjustments to compare the performance of several ML methods. To address this challenge, we first harness the internal ratings-based approach to identify up to 13 risk components that we classify into 3 main categories—statistics, technology, and market conduct. Second, to evaluate the importance of each risk category, we collect a series of regulatory documents related to three potential use cases—regulatory capital, credit scoring, or provisioning—and we compute the weight of each category according to the intensity of their mentions, using natural language processing and a risk terminology based on expert knowledge. Finally, we test our framework using popular ML models in credit risk, and a publicly available database, to quantify some proxies of a subset of risk factors that we deem representative. We measure the statistical risk according to the number of hyperparameters and the stability of the predictions. The technological risk is assessed through the transparency of the algorithm and the latency of the ML training method, while the market conduct risk is quantified by the time it takes to run a post hoc technique (SHapley Additive exPlanations) to interpret the output.

Keywords: Artificial intelligence; Machine learning; Credit risk; Interpretability; Bias; Internal ratings based model; IRB model; Natural language processing; NLP (search for similar items in EconPapers)
Date: 2022
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (6)

Downloads: (external link)
http://link.springer.com/10.1186/s40854-022-00366-1 Abstract (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:fininn:v:8:y:2022:i:1:d:10.1186_s40854-022-00366-1

Ordering information: This journal article can be ordered from
http://www.springer. ... nomics/journal/40589

DOI: 10.1186/s40854-022-00366-1

Access Statistics for this article

Financial Innovation is currently edited by J. Leon Zhao and Zongyi

More articles in Financial Innovation from Springer, Southwestern University of Finance and Economics
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().