The Challenger: When Do New Data Sources Justify Switching Machine Learning Models?

Christophe Pérignon, Sébastien Saurin, Flore Sentenac and Vassilis Digalakis Jr
Additional contact information
Christophe Pérignon: HEC Paris - Finance Department
Sébastien Saurin: HEC Paris
Flore Sentenac: HEC Paris
Vassilis Digalakis Jr: Boston University

No 1601, HEC Research Papers Series from HEC Paris

Abstract: We study the problem of deciding whether, and when an organization should replace a trained incumbent model with a challenger relying on newly available features. We develop a unified economic and statistical framework that links learning-curve dynamics, data-acquisition and retraining costs, and discounting of future gains. First, we characterize the optimal switching time in stylized settings and derive closed-form expressions that quantify how horizon length, learning-curve curvature, and cost differentials shape the optimal decision. Second, we propose three practical algorithms: a one-shot baseline, a greedy sequential method, and a look-ahead sequential method. Using a real-world credit-scoring dataset with gradually arriving alternative data, we show that (i) optimal switching times vary systematically with cost parameters and learning-curve behavior, and (ii) the look-ahead sequential method outperforms other methods and is able to approach in value an oracle with full foresight. Finally, we establish finite-sample guarantees, including conditions under which the sequential look-ahead method achieve sublinear regret relative to that oracle. Our results provide an operational blueprint for economically sound model transitions as new data sources become available.

Keywords: Model Transition; Alternative Data; Credit Scoring; AI Operationalization; Trustworthy AI And Machine Learning (search for similar items in EconPapers)
JEL-codes: C53 C61 G21 (search for similar items in EconPapers)
Pages: 71 pages
Date: 2025-12-25
References: Add references at CitEc
Citations:

Downloads: (external link)
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=5946174 Full text (text/html)
Our link check indicates that this URL is bad, the error code is: 403 Forbidden

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:ebg:heccah:1601

DOI: 10.2139/ssrn.5946174

Access Statistics for this paper

More papers in HEC Research Papers Series from HEC Paris HEC Paris, 1 Rue de la Libération, 78350 Jouy-en-Josas, France. Contact information at EDIRC.
Bibliographic data for series maintained by Antoine Haldemann ( this e-mail address is bad, please contact ).