UCB-Type Learning Algorithms with Kaplan–Meier Estimator for Lost-Sales Inventory Models with Lead Times

Chengyi Lyu (), Huanan Zhang () and Linwei Xin ()
Additional contact information
Chengyi Lyu: Leeds School of Business, University of Colorado Boulder, Boulder, Colorado 80309
Huanan Zhang: Leeds School of Business, University of Colorado Boulder, Boulder, Colorado 80309
Linwei Xin: Booth School of Business, University of Chicago, Chicago, Illinois 60637

Operations Research, 2024, vol. 72, issue 4, 1317-1332

Abstract: In this paper, we consider a classic periodic-review lost-sales inventory system with lead times, which is notoriously challenging to optimize with a wide range of real-world applications. We consider a joint learning and optimization problem in which the decision maker does not know the demand distribution a priori and can only use past sales information (i.e., censored demand). Departing from existing learning algorithms on this learning problem that require the convexity property of the underlying system, we develop an upper confidence bound (UCB)-type learning framework that incorporates simulations with the Kaplan–Meier estimator and demonstrate its applicability to learning not only the optimal capped base-stock policy in which convexity no longer holds, but also the optimal base-stock policy with a regret that matches the best existing result. Compared with a classic multi-armed bandit problem, our problem has unique challenges because of the nature of the inventory system, because (1) each action has long-term impacts on future costs, and (2) the system state space is exponentially large in the lead time. As such, our learning algorithms are not naive adoptions of the classic UCB algorithm; in fact, the design of the simulation steps with the Kaplan–Meier estimator and averaging steps is novel in our algorithms, and the confidence width in the UCB index is also different from the classic one. We prove the regrets of our learning algorithms are tight up to a logarithmic term in the planning horizon T . Our extensive numerical experiments suggest the proposed algorithms (almost) dominate existing learning algorithms. We also demonstrate how to select which learning algorithm to use with limited demand data.

Keywords: Operations and Supply Chains; lost sales; censored demand; learning algorithm; capped base-stock policy; Kaplan–Meier estimator (search for similar items in EconPapers)
Date: 2024
References: Add references at CitEc
Citations:

Downloads: (external link)
http://dx.doi.org/10.1287/opre.2022.0273 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:inm:oropre:v:72:y:2024:i:4:p:1317-1332

Access Statistics for this article

More articles in Operations Research from INFORMS Contact information at EDIRC.
Bibliographic data for series maintained by Chris Asher ().