An Improved Transformer Framework for Well-Overflow Early Detection via Self-Supervised Learning

Wan Yi, Wei Liu, Jiasheng Fu, Lili He and Xiaosong Han ()
Additional contact information
Wan Yi: Key Laboratory for Symbol Computation and Knowledge Engineering of National Education Ministry, College of Computer Science and Technology, Jilin University, Changchun 130012, China
Wei Liu: CNPC Engineering Technology R&D Company Limited, National Engineering Research Center of Oil & Gas Drilling and Completion Technology, Beijing 102206, China
Jiasheng Fu: CNPC Engineering Technology R&D Company Limited, National Engineering Research Center of Oil & Gas Drilling and Completion Technology, Beijing 102206, China
Lili He: Key Laboratory for Symbol Computation and Knowledge Engineering of National Education Ministry, College of Computer Science and Technology, Jilin University, Changchun 130012, China
Xiaosong Han: Key Laboratory for Symbol Computation and Knowledge Engineering of National Education Ministry, College of Computer Science and Technology, Jilin University, Changchun 130012, China

Energies, 2022, vol. 15, issue 23, 1-12

Abstract: Oil drilling has always been considered a vital part of resource exploitation, and during which overflow is the most common and tricky threat that may cause blowout, a catastrophic accident. Therefore, to prevent further damage, it is necessary to detect overflow as early as possible. However, due to the unbalanced distribution and the lack of labeled data, it is difficult to design a suitable solution. To address this issue, an improved Transformer Framework based on self-supervised learning is proposed in this paper, which can accurately detect overflow 20 min in advance when the labeled data are limited and severely imbalanced. The framework includes a self-supervised pre-training scheme, which focuses on long-term time dependence that offers performance benefits over fully supervised learning on downstream tasks and makes unlabeled data useful in the training process. Next, to better extract temporal features and adapt to multi-task training process, a Transformer-based auto-encoder with temporal convolution layer is proposed. In the experiment, we used 20 min data to detect overflow in the next 20 min. The results show that the proposed framework can reach 98.23% accuracy and 0.84 F1 score, which is much better than other methods. We also compare several modifications of our framework and different pre-training tasks in the ablation experiment to prove the advantage of our methods. Finally, we also discuss the influence of important hyperparameters on efficiency and accuracy in the experiment.

Keywords: oil drilling; overflow; time series prediction; abnormal detection; self-supervised learning; transformer (search for similar items in EconPapers)
JEL-codes: Q Q0 Q4 Q40 Q41 Q42 Q43 Q47 Q48 Q49 (search for similar items in EconPapers)
Date: 2022
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/1996-1073/15/23/8799/pdf (application/pdf)
https://www.mdpi.com/1996-1073/15/23/8799/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jeners:v:15:y:2022:i:23:p:8799-:d:980479

Access Statistics for this article

Energies is currently edited by Ms. Agatha Cao

More articles in Energies from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().