Explainable Performance

Sullivan Hué, Christophe Hurlin, Christophe Pérignon and Sébastien Saurin
Additional contact information
Christophe Pérignon: HEC Paris
Sébastien Saurin: University of Orleans, Laboratoire d'économie d'Orléans, Students

No 1463, HEC Research Papers Series from HEC Paris

Abstract: In credit scoring, machine learning models are known to outperform standard parametric models. As they condition access to credit, banking supervisors and internal model validation teams need to monitor their predictive performance and to identify the features with the highest impact on performance. To facilitate this, we introduce the XPER methodology to decompose a performance metric (e.g., AUC, R^2) into specific contributions associated with the various features of a classification or regression model. XPER is theoretically grounded on Shapley values and is both model-agnostic and performance metric-agnostic. Furthermore, it can be implemented either at the model level or at the individual level. Using a novel dataset of car loans, we decompose the AUC of a machine-learning model trained to forecast the default probability of loan applicants. We show that a small number of features can explain a surprisingly large part of the model performance. Furthermore, we find that the features that contribute the most to the predictive performance of the model may not be the ones that contribute the most to individual forecasts (SHAP). We also show how XPER can be used to deal with heterogeneity issues and significantly boost out-of-sample performance.

Keywords: Machine learning; Explainability; Performance metric; Shapley value (search for similar items in EconPapers)
JEL-codes: C40 C52 (search for similar items in EconPapers)
Pages: 73 pages
Date: 2022-11-22
New Economics Papers: this item is included in nep-big and nep-cmp
References: Add references at CitEc
Citations:

Downloads: (external link)
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4280563 Full text (text/html)

Related works:
Working Paper: Explainable Performance (2022)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:ebg:heccah:1463

DOI: 10.2139/ssrn.4280563

Access Statistics for this paper

More papers in HEC Research Papers Series from HEC Paris HEC Paris, 78351 Jouy-en-Josas cedex, France. Contact information at EDIRC.
Bibliographic data for series maintained by Antoine Haldemann ().