The Challenger: When Do New Data Sources Justify Switching Machine Learning Models?

Vassilis Digalakis Jr, Christophe Pérignon, Sébastien Saurin and Flore Sentenac ()
Additional contact information
Vassilis Digalakis Jr: BU - Boston University [Boston]
Christophe Pérignon: HEC Paris - Ecole des Hautes Etudes Commerciales
Sébastien Saurin: HEC Paris - Ecole des Hautes Etudes Commerciales
Flore Sentenac: HEC Paris - Ecole des Hautes Etudes Commerciales

Working Papers from HAL

Abstract: We study the problem of deciding whether, and when an organization should replace a trained incumbent model with a challenger relying on newly available features. We develop a unified economic and statistical framework that links learning-curve dynamics, data-acquisition and retraining costs, and discounting of future gains. First, we characterize the optimal switching time in stylized settings and derive closed-form expressions that quantify how horizon length, learning-curve curvature, and cost differentials shape the optimal decision. Second, we propose three practical algorithms: a one-shot baseline, a greedy sequential method, and a look-ahead sequential method. Using a real-world credit-scoring dataset with gradually arriving alternative data, we show that (i) optimal switching times vary systematically with cost parameters and learning-curve behavior, and (ii) the look-ahead sequential method outperforms other methods and is able to approach in value an oracle with full foresight. Finally, we establish finite-sample guarantees, including conditions under which the sequential look-ahead method achieve sublinear regret relative to that oracle. Our results provide an operational blueprint for economically sound model transitions as new data sources become available.

Keywords: Model Transition; Alternative Data; Credit Scoring; AI Operationalization; Trustworthy AI And Machine Learning (search for similar items in EconPapers)
Date: 2025-12-19
References: Add references at CitEc
Citations:

There are no downloads for this item, see the EconPapers FAQ for hints about obtaining it.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:hal:wpaper:hal-05562736

DOI: 10.2139/ssrn.5946174

Access Statistics for this paper

More papers in Working Papers from HAL
Bibliographic data for series maintained by CCSD ().