Leveraging Large Language Models and BERT for Log Parsing and Anomaly Detection

Yihan Zhou, Yan Chen, Xuanming Rao, Yukang Zhou, Yuxin Li and Chao Hu ()
Additional contact information
Yihan Zhou: School of Computer Science and Engineering, Central South University, Changsha 410083, China
Yan Chen: Logistics Department, Central South University, Changsha 410083, China
Xuanming Rao: School of Computer Science and Engineering, Central South University, Changsha 410083, China
Yukang Zhou: Department of Electrical and Information Engineering, Hong Kong Polytechnic University, Hong Kong, China
Yuxin Li: School of Computer Science and Engineering, Nanyang Technological University, Singapore 639798, Singapore
Chao Hu: School of Electronic Information, Central South University, Changsha 410083, China

Mathematics, 2024, vol. 12, issue 17, 1-20

Abstract: Computer systems and applications generate large amounts of logs to measure and record information, which is vital to protect the systems from malicious attacks and useful for repairing faults, especially with the rapid development of distributed computing. Among various logs, the anomaly log is beneficial for operations and maintenance (O&M) personnel to locate faults and improve efficiency. In this paper, we utilize a large language model, ChatGPT, for the log parser task. We choose the BERT model, a self-supervised framework for log anomaly detection. BERT, an embedded transformer encoder, with a self-attention mechanism can better handle context-dependent tasks such as anomaly log detection. Meanwhile, it is based on the masked language model task and next sentence prediction task in the pretraining period to capture the normal log sequence pattern. The experimental results on two log datasets show that the BERT model combined with an LLM performed better than other classical models such as Deelog and Loganomaly.

Keywords: BERT; anomaly log detection; transformer; self-attention; large language models (search for similar items in EconPapers)
JEL-codes: C (search for similar items in EconPapers)
Date: 2024
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/2227-7390/12/17/2758/pdf (application/pdf)
https://www.mdpi.com/2227-7390/12/17/2758/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jmathe:v:12:y:2024:i:17:p:2758-:d:1472358

Access Statistics for this article

Mathematics is currently edited by Ms. Emma He

More articles in Mathematics from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().