A deep reinforcement learning assisted simulated annealing algorithm for a maintenance planning problem

Fuat Kosanoglu, Mahir Atmis and Hasan Hüseyin Turan ()
Additional contact information
Fuat Kosanoglu: Yalova University
Mahir Atmis: Yalova University
Hasan Hüseyin Turan: University of New South Wales

Annals of Operations Research, 2024, vol. 339, issue 1, No 4, 79-110

Abstract: Abstract Maintenance planning aims to improve the reliability of assets, prevent the occurrence of asset failures, and reduce maintenance costs associated with downtime of assets and maintenance resources (such as spare parts and workforce). Thus, effective maintenance planning is instrumental in ensuring high asset availability with the minimum cost. Nevertheless, to find such optimal planning is a nontrivial task due to the (i) complex and usually nonlinear inter-relationship between different planning decisions (e.g., inventory level and workforce capacity), and (ii) stochastic nature of the system (e.g., random failures of parts installed in assets). To alleviate these challenges, we study a joint maintenance planning problem by considering several decisions simultaneously, including workforce planning, workforce training, and spare parts inventory management. We develop a hybrid solution algorithm ( $$\mathcal {DRLSA}$$ DRLSA ) that is a combination of Double Deep Q-Network based Deep Reinforcement Learning (DRL) and Simulated Annealing (SA) algorithms. In each episode of the proposed algorithm, the best solution found by DRL is delivered to SA to be used as an initial solution, and the best solution of SA is delivered to DRL to be used as the initial state. Different from the traditional SA algorithms where neighborhood structures are selected only randomly, the DRL part of $$\mathcal {DRLSA}$$ DRLSA learns to choose the best neighborhood structure to use based on experience gained from previous episodes. We compare the performance of the proposed solution algorithm with several well-known meta-heuristic algorithms, including Simulated Annealing, Genetic Algorithm (GA), and Variable Neighborhood Search (VNS). Further, we also develop a Machine Learning (ML) algorithm (i.e., K-Median) as another benchmark in which different properties of spare parts (e.g., failure rates, holding costs, and repair rates) are used as clustering features for the ML algorithm. Our study reveals that the $$\mathcal {DRLSA}$$ DRLSA finds the optimal solutions for relatively small-size instances, and it has the potential to outperform traditional meta-heuristic and ML algorithms.

Keywords: Maintenance planning; Workforce planning and training; Inventory management; Double deep Q-network; Deep reinforcement learning; Simulated annealing (search for similar items in EconPapers)
Date: 2024
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://link.springer.com/10.1007/s10479-022-04612-8 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:annopr:v:339:y:2024:i:1:d:10.1007_s10479-022-04612-8

Ordering information: This journal article can be ordered from
http://www.springer.com/journal/10479

DOI: 10.1007/s10479-022-04612-8

Access Statistics for this article

Annals of Operations Research is currently edited by Endre Boros

More articles in Annals of Operations Research from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().