Deep reinforcement learning for joint optimization of maintenance and spare parts ordering considering spare parts supply uncertainty

Yunxin Zhu, Meimei Zheng, Zhiyun Su, Tangbin Xia, Jie Lin and Ershun Pan

Reliability Engineering and System Safety, 2025, vol. 264, issue PB

Abstract: Efficient maintenance and spare parts ordering strategies can reduce costs for manufacturing companies. In recent years, important components may suffer supply risks due to geopolitical conflicts, trade conflicts, and limitations of key resources. This paper investigates the joint optimization of condition-based maintenance and dual sourcing of spare parts from reliable and unreliable suppliers. We formulate this joint decision problem with a Markov decision process and design a value iteration algorithm to obtain exact solutions for the optimal maintenance and ordering policy. However, the value iteration algorithm is not suitable for solving large-scale problems due to its long running time. Thus, we develop a deep Q-network (DQN) algorithm based on deep reinforcement learning to improve computation efficiency. Numerical experiments are conducted to validate the effectiveness of the DQN algorithm. The results show that the DQN algorithm can reduce the running time by 92.58 % for systems with more than 4 components and more than 5 states within a 4.82 % cost gap compared to the value iteration algorithm. Compared to the separate heuristic policy, the DQN algorithm can averagely reduce the cost by 11.27 %.

Keywords: Condition-based maintenance; Supply uncertainty; Joint optimization; Deep reinforcement learning (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0951832025005861
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:reensy:v:264:y:2025:i:pb:s0951832025005861

DOI: 10.1016/j.ress.2025.111385

Access Statistics for this article

Reliability Engineering and System Safety is currently edited by Carlos Guedes Soares

More articles in Reliability Engineering and System Safety from Elsevier
Bibliographic data for series maintained by Catherine Liu ().