Bandit-based inventory optimisation: Reinforcement learning in multi-echelon supply chains

Deniz Preil and Michael Krapp

International Journal of Production Economics, 2022, vol. 252, issue C

Abstract: Even though base-stock policies are per se straightforward, determining them in complex, stochastic multi-echelon supply chains is often cumbersome or even analytically impossible. Therefore, a wide range of heuristics has been proposed for this purpose. This is the first study considering the problem as a multi-armed bandit problem. In this context, we investigate two algorithms: first, we propose an approach that is based on upper confidence bounds and priority queues. This so-called PQ-UCB algorithm allows us to drastically reduce the runtime of upper confidence bound allocation strategies in problems with large action spaces. Subsequently, we apply the parameter-free sequential halving (SH) algorithm. We investigate various scenarios to compare the performance of both algorithms with the performance of a genetic algorithm and a simulated annealing algorithm taken from the literature. PQ-UCB as well as SH outperform both benchmark metaheuristics and require substantially less effort related to parameter tuning (or even no effort in the case of SH). As multi-armed bandits are not common in inventory optimisation so far, we aim to emphasise their strengths and hope to promote their dissemination also in other domains of supply chain management.

Keywords: Multi-armed bandits; Inventory optimisation; Reinforcement learning; Supply chain management; Simulation optimisation (search for similar items in EconPapers)
Date: 2022
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (3)

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0925527322001670
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:proeco:v:252:y:2022:i:c:s0925527322001670

DOI: 10.1016/j.ijpe.2022.108578

Access Statistics for this article

International Journal of Production Economics is currently edited by Stefan Minner

More articles in International Journal of Production Economics from Elsevier
Bibliographic data for series maintained by Catherine Liu ().