An integrated multi-head dual sparse self-attention network for remaining useful life prediction

Jiusi Zhang, Xiang Li, Jilun Tian, Hao Luo and Shen Yin

Reliability Engineering and System Safety, 2023, vol. 233, issue C

Abstract: Committed to accident prevention, prediction of remaining useful life (RUL) plays a crucial role in prognostics health management technology. Conventional convolutional neural network and long-short-term memory network have notable limitations in the size of convolution in processing temporal data and the associations between non-adjacent data when predicting the RUL, respectively. Although the proposal of the Transformer provides an opportunity to solve the shortcomings mentioned above, Transformer still has some limitations. Precisely, the Transformer model awaits in-depth research focusing on vital local regions and decreasing computational complexity. In this sense, this paper proposes a novel integrated multi-head dual sparse self-attention network (IMDSSN) based on a modified Transformer to predict the RUL. From two sparse perspectives, the proposed IMDSSN includes a multi-head ProbSparse self-attention network (MPSN) and a multi-head LogSparse self-attention network (MLSN). Specifically, MPSN is designed to filter out the primary function of the dot product operation, thereby improving computational efficiency. Furthermore, considering the data inside the whole time window, a comprehensive logarithmic-based sparse strategy in MLSN is proposed to reduce the amount of computation. An aircraft turbofan engine dataset is used to verify the proposed IMDSSN, which demonstrates that the IMDSSN is better than some conventional approaches.

Keywords: Multi-head self-attention; Remaining useful life; Transformer; Sparse strategy; Prediction (search for similar items in EconPapers)
Date: 2023
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (13)

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S095183202300011X
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:reensy:v:233:y:2023:i:c:s095183202300011x

DOI: 10.1016/j.ress.2023.109096

Access Statistics for this article

Reliability Engineering and System Safety is currently edited by Carlos Guedes Soares

More articles in Reliability Engineering and System Safety from Elsevier
Bibliographic data for series maintained by Catherine Liu ().