Fighting sampling bias: A framework for training and evaluating credit scoring models

Nikita Kozodoi, Stefan Lessmann, Morteza Alamgir, Luis Moreira-Matias and Konstantinos Papakonstantinou

European Journal of Operational Research, 2025, vol. 324, issue 2, 616-628

Abstract: Scoring models support decision-making in financial institutions. Their estimation and evaluation rely on labeled data from previously accepted clients. Ignoring rejected applicants with unknown repayment behavior introduces sampling bias, as the available labeled data only partially represents the population of potential borrowers. This paper examines the impact of sampling bias and introduces new methods to mitigate its adverse effect. First, we develop a bias-aware self-labeling algorithm for scorecard training, which debiases the training data by adding selected rejects with an inferred label. Second, we propose a Bayesian framework to address sampling bias in scorecard evaluation. To provide reliable projections of future scorecard performance, we include rejected clients with random pseudo-labels in the test set and use Monte Carlo sampling to estimate the scorecard’s expected performance across label realizations. We conduct extensive experiments using both synthetic and observational data. The observational data includes an unbiased sample of applicants accepted without scoring, representing the true borrower population and facilitating a realistic assessment of reject inference techniques. The results show that our methods outperform established benchmarks in predictive accuracy and profitability. Additional sensitivity analysis clarifies the conditions under which they are most effective. Comparing the relative effectiveness of addressing sampling bias during scorecard training versus evaluation, we find the latter much more promising. For example, we estimate the expected return per dollar issued to increase by up to 2.07 and up to 5.76 percentage points when using bias-aware self-labeling and Bayesian evaluation, respectively.

Keywords: OR in banking; Machine learning; Credit scoring; Sampling bias; Reject inference (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0377221725000839
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:ejores:v:324:y:2025:i:2:p:616-628

DOI: 10.1016/j.ejor.2025.01.040

Access Statistics for this article

European Journal of Operational Research is currently edited by Roman Slowinski, Jesus Artalejo, Jean-Charles. Billaut, Robert Dyson and Lorenzo Peccati

More articles in European Journal of Operational Research from Elsevier
Bibliographic data for series maintained by Catherine Liu ().