Polo: Adaptive Trie-Based Log Parser for Anomaly Detection

Yuezhou Zhou and Yuxin Su ()
Additional contact information
Yuezhou Zhou: School of Software Engineering, Sun Yat-sen University, Zhuhai 528406, China
Yuxin Su: School of Software Engineering, Sun Yat-sen University, Zhuhai 528406, China

Mathematics, 2023, vol. 11, issue 23, 1-14

Abstract: Automated log parsing is essential for many log-mining applications, as logs provide a vast range of information on events and variations within an operating system or software at runtime. Over the years, various methods have been proposed for log parsing. With improved log-parsing methods, log-mining applications can gain deeper insights into system behaviors and identify anomalies or failures promptly. However, current log parsers still face limitations, such as insufficient parsing of log templates and a lack of parallelism, as well as inaccurate log template parsing. To overcome these limitations, we have designed Polo, a parser that leverages a prefix forest composed of ternary search trees to mine templates from logs. We then conducted extensive experiments to evaluate the accuracy of Polo on nine representative system logs, achieving an average accuracy of 0.987. It is 9.93% to 40.95% faster than the state-of-the-art parsing methods. Furthermore, we evaluated our approach on a downstream log analysis task, specifically anomaly detection. The experimental results demonstrated that, in terms of F1-score, our parser outperformed Deeplog, LogAnomaly, CNN, and LogRobust by 11.5%, 4%, 1%, and 19.1%, respectively, exhibiting a promising recall score of 0.971. These results indicate the effectiveness of Polo for anomaly detection.

Keywords: log analysis; parsing; trie; log anomaly; software engineering (search for similar items in EconPapers)
JEL-codes: C (search for similar items in EconPapers)
Date: 2023
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/2227-7390/11/23/4797/pdf (application/pdf)
https://www.mdpi.com/2227-7390/11/23/4797/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jmathe:v:11:y:2023:i:23:p:4797-:d:1289234

Access Statistics for this article

Mathematics is currently edited by Ms. Emma He

More articles in Mathematics from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().