A soft actor-critic reinforcement learning-based method for remaining useful life prediction

Shousheng Ding, Lei Meng, Jie Shang, Chen Jiang, Haobo Qiu and Liang Gao

Reliability Engineering and System Safety, 2025, vol. 261, issue C

Abstract: Remaining useful life (RUL) prediction techniques play a crucial role in manufacturing equipment condition management and maintenance planning. Currently, data-driven deep learning methods have made significant advancements in this field. However, traditional approaches have not adequately considered the temporal correlations in both sensor data and RUL prediction values during the degradation process of equipment. The existing reinforcement learning (RL) methods face challenges such as lacking of sufficient lifespan variation information in the state variables, ignorance of dynamic changes in prediction error in the reward function design, and adoption of fixed interaction termination conditions that can't effectively promote the agent's learning of device degradation information. Therefore, this paper proposes a RL model based on the soft actor-critic (SAC) algorithm. Firstly, an autoencoder is employed to extract key features from the data collected by sensors. Subsequently, these key features, along with multi-dimensional lifespan features containing information from multiple historical time steps, are utilized to construct the state variables in RL. Next, a reward function is formulated taking into account error gradients. Finally, a progressive early stopping method is proposed to train the model. Extensive experiments are conducted on the CMAPSS dataset and XJTU-SY bearing dataset, and the proposed method demonstrates higher prediction accuracy compared to mainstream approaches.

Keywords: Deep reinforcement learning; Remaining useful life; Progressive early stopping; Soft Actor-Critic (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0951832025003229
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:reensy:v:261:y:2025:i:c:s0951832025003229

DOI: 10.1016/j.ress.2025.111121

Access Statistics for this article

Reliability Engineering and System Safety is currently edited by Carlos Guedes Soares

More articles in Reliability Engineering and System Safety from Elsevier
Bibliographic data for series maintained by Catherine Liu ().